A Biometrics-embedded System

Based on Haptics for User

Authentication in Virtual
Environments
Presented by Mauricio Orozco

October 2007

University
Of Ottawa
SITE/EITI

http://www.mcrlab.uottawa.ca MCRLab
Outline

• Introduction
• Contributions
• Research Statement
• Outline Architecture
• Methodology
• Conclusions
• Future Work

http://www.mcrlab.uottawa.ca MCRLab
Contributions
–Design and development of a procedure for:
•Combining haptic technology to discriminate human-hand psychomotor
patterns to be used for authentication/verification purposes

–Analysis and definition of features and feature set

•Haptic data to be used as biometric identifiers to characterize the human

Interaction behaviour
•The amount of information contained in an individual’s features characterizes
(i.e. Force and Torque )the uniqueness of an individual
•The user behavior exhibited in Virtual Domains should be modeled as
“entropic signatures” according to features that are consistently unique
for each user

−Results suggest that haptic-based biometrics are best suited to verification

•At 25% False Acceptance Rate (FAR), the Probability of Verification (PV) for
haptic-based Application (i.e. “maze”) identification was at best 75.5%,
While the PV of verification system was at best 98%

http://www.mcrlab.uottawa.ca MCRLab
 Publications from this thesis
Papers in refereed Journals

1. Mohamad Eid, Mauricio Orozco and Abdulmotaleb El Saddik, "A Guided Tour in Haptic Audio Visual Environment and Applications",
Int. J. of Advanced Media and Communication. Vol.1 (3), 2007
2. Abdulmotaleb El Saddik, Mauricio Orozco, Yednek Asfaw, Shervin Shirmohammadi and Andy Adler, "A Novel Biometric System for
Identification and Verification of Haptic Users", IEEE Transactions on Instrumentation and Measurement, vol.56 (3) 2007.
3. Mauricio Orozco, Matthew Graydon, Shervin Shirmohammadi, and Abdulmotaleb El Saddik, "Experiments in Haptic-Based
Authentication of Humans", Journal of Multimedia Tools and Applications, 2007
4. Mauricio Orozco and Abdulmotaleb El Saddik “AdHapticA” IEEE Transactions on Instrumentation and Measurement, accepted to
appear in 2008

Papers in refereed Conferences

1. Mauricio Orozco, Matthew Graydon, Shervin Shirmohammadi and Abdulmotaleb El Saddik, "Using Haptic Interfaces for User
Verification in Virtual Environments", In proceedings of the 2006 IEEE International Conference on Virtual Environments, Human-
Computer Interfaces, and Measurement Systems, Spain, July 2006.
2. Ismail Shakra, Mauricio Orozco, Abdulmotaleb El Saddik, Shervin Shirmohammadi, Edward Lemaire, "VR-Based Hand Rehabilitation
using a Haptic-Based Framework", In proceedings of the 2006 IEEE Instrumentation and Measurement Technology Conference
(IMTC/06), Sorrento, Italy, 24 - 27 April 2006.
3. Ismail Shakra, Mauricio Orozco, Abdulmotaleb El Saddik, Shervin Shirmohammadi, Edward Lemaire, "Haptic Instrumentation for
Physical Rehabilitation of Stroke Patients", In proceedings of the 2006 IEEE International Workshop on Medical Measurement and
Applications Benevento, Italy 20 - 21 April 2006.
4. Mauricio Orozco, Yednek Asfaw, Shervin Shirmohammadi, Andy Adler, and Abdulmotaleb El Saddik, "Haptic-Based Biometrics: A
Feasibility Study", In Proceedings of the IEEE VR 2006, pp. 265 - 271, March 25-29, Alexandria, Virginia, USA, 2006.
5. Yednek Asfaw, Mauricio Orozco, Shervin Shirmohammadi, Abdulmotaleb El Saddik, Andy Adler, "Participant Identification in Haptic
systems using HMMs", In Proceedings of the fourth IEEE International Workshop on Haptic Virtual Environments and their
Applications (HAVE2005)", pp. 127 -132 2005.
6. Mauricio Orozco and Abdulmotaleb El Saddik, "Haptic: The New Biometrics-embedded Media to Recognizing and Quantifying Human
Patterns" In proceedings of 13th Annual ACM International Conference on Multimedia (ACM-MM 2005), Singapore, November 06-12,
2005.
7. Mauricio Orozco , Ismail Shakra and Abdulmotaleb El Saddik, “Adaptive Haptic Framework”, In proceedings of the IEEE International
Conference on Virtual Environments, Human-Computer Interfaces, and Measurement Systems, Giardini Naxos, Italy, 18-20 July
2005.
8. Mauricio Orozco and Abdulmotaleb El Saddik, “Signature Identification with Haptic devices”, In proceedings of the IEEE International
Conference on Virtual Environments, Human-Computer Interfaces, and Measurement Systems, Giardini Naxos, Italy, 18-20 July
2005.
9. Mauricio Orozco, Yednek Asfaw, Andy Adler, Shervin Shirmohammadi, and Abdulmotaleb El Saddik, “Automatic Identification of
Participants in Haptic Systems”, In proceedings of the 2005 IEEE Instrumentation and Measurement Technology Conference
(IMTC/05), Ottawa, Ontario, Canada, 17-19 May 2005.

http://www.mcrlab.uottawa.ca MCRLab
 Motivation

Haptic Technology & Apps

Telepresence +Haptics

Internet
Haptics Telephony

noi t car et nI f ol eve L

Letters

Time

Biometric Technology & Apps

Biometrics

Source: International Biometric Group, New York, NY

http://www.mcrlab.uottawa.ca MCRLab
Why Use Haptics?

Time

2D Position

3D Position

Force

Pressure

orientation
Angular

Torque

Velocity
Keyboard

Mouse

CyberGlove

Digital Tablet

Haptics

http://www.mcrlab.uottawa.ca MCRLab
Introduction

Haptic Systems

Biometrics Systems

http://www.mcrlab.uottawa.ca MCRLab
Research Statement

 H1: The quantitative performance of specific haptic interface lies significantly

in acceptable range to be considered as biometric identifier system

 H0: The quantitative performance of specific haptic interface does not lie
significantly in acceptable range to be considered as biometric identifier
system

http://www.mcrlab.uottawa.ca MCRLab
Related Work: Behavioral Biometrics

Dynamic Signature Verification

[Fernandez et al, 2005],[Plamondon, 1990], …plenty
Be
ha
v io
ur
al Keystroke Dynamics
Bi
om
et
ri cs [ Joyce and Gupta , 1990]
[ Umpresh and Williams, 1985]
[Obaidat and Sadoun,1997] and more

Mouse :DSV

CyberGlove:DataGlove [Everitt and McOwan, 2003]

[Everitt and McOwan, 2003]

http://www.mcrlab.uottawa.ca MCRLab
Proposed Approach: AdHapticA

Methodology

Adaptive Haptic Framework Feature Generation

/Selection
Haptic -Based Applications Classifier Design / Evaluation
+ Virtual Maze
+ Virtual Cheque Feature Generation :
+ Virtual Phone + K-means
x = { Vx, Vy, Vz,...}
+ Neural Networks
+ PCA
Preprocessing + DTW
+Nearest Neighbor

:
Feature Selection System Evaluation
+ Relative Entropy Matching Score

Behavioural DAO

Data Acquisition Repository

http://www.mcrlab.uottawa.ca MCRLab
Proposed Approach: AdHapticA

Methodology

Adaptive Haptic Framework Feature Generation

/Selection
Haptic -Based Applications Classifier Design / Evaluation
+ Virtual Maze
+ Virtual Cheque Feature Generation :
+ Virtual Phone + K-means
x = { Vx, Vy, Vz,...}
+ Neural Networks
+ PCA
Preprocessing + DTW
+Nearest Neighbor

:
Feature Selection System Evaluation
+ Relative Entropy Matching Score

Behavioural DAO

Data Acquisition Repository

http://www.mcrlab.uottawa.ca MCRLab
Haptic-Based Applications
Applications
Single Point Interaction

PHANToM Haptic Desktop

Multiple Point Interaction CyberGrasp Unit

http://www.mcrlab.uottawa.ca MCRLab
Proposed Approach: AdHapticA

Methodology

Adaptive Haptic Framework Feature Generation

/Selection
Haptic -Based Applications Classifier Design / Evaluation
+ Virtual Maze
+ Virtual Cheque Feature Generation :
+ Virtual Phone + K-means
x = { Vx, Vy, Vz,...}
+ Neural Networks
+ PCA
Preprocessing + DTW
+Nearest Neighbor

:
Feature Selection System Evaluation
+ Relative Entropy Matching Score

Behavioural DAO

Data Acquisition Repository

http://www.mcrlab.uottawa.ca MCRLab
Data Acquisition

Database consists ~ 109 volunteers

( > 2 year )

Each providing 10 genuine samples:

+ Handwritten signature
+ Maze solved
+ Dialled telephone codes

Participants were given the opportunity

to practice each application before

Each process recorded among others

parameters the pen’s position(x,y,z), force
applied (N) and device angle (φ)

http://www.mcrlab.uottawa.ca MCRLab
Analysis
DTW( Dynamic Time Warping
Functional Approach

Spectral Analysis (FFT)

Single Point Interaction

K-Means
Parametric Approach

HAPTIC DEVICE Majority Class.

Multiple Point Interaction

Euclidean Distance

Neural Networks
Information Content

http://www.mcrlab.uottawa.ca MCRLab
Functional Approach
3 N
MS =∑ ∑( d c1,i −lc2,i (t p )) 2
c =1 i =1
Spectral
Analysis:FFT

Validation
Scheme
Maze

Dynamic Performance
Time Warping
Equal Error Rate
(EER)22.3% with a
threshold Matching Score
of 0.195

http://www.mcrlab.uottawa.ca MCRLab
Parametric Approach

Virtual Feature Set: 35

Cheque Variables Pe rform ance of Class ifie r
1

0.9
0.8

0.7
0.6

0.5

0.4

0.3
0.2

0.1

Validation Scheme 0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

FA R e r r or r at e

Maze Virtual Check Maze Virtual Phone

Performance

Equal Error Rate (EER) of

Majority Rule 6.0% with a threshold of
Virtual Mobile
Phone 1.6 for the best (Cheque )

http://www.mcrlab.uottawa.ca MCRLab
Analysis of the Haptic Information Content

X = [v x , v y , v z , Fx , Fy , Fz ,τ x ,τ y ,τ z ,θ ]T

To measure the uncertainty

associated with a random variable

p( x)
D( p || q) = ∑ p ( x ) log
x∈ X q( x)

• Advantages of Using Relative Entropy

– Quantifies the information contained in a piece of data: in bits (if using base-2
logarithms)
– Another distinct advantage of using relative entropy as a measure of
biometric feature uniqueness is that it accounts for both the mean and
variance of a distribution
– The Relative or Shannon entropy is a measure of the average information
content the recipient

http://www.mcrlab.uottawa.ca MCRLab
Analysis :Information Content
The information content of biometric
features differ amongst users of the 80 Relative Entropy of User Features

haptic 70 60

60
50 xy Torque

Relative Entropy [bits]

50 3D Torque
xy Torque
40 Force
40 xyz Torque
Position Force

Entropic signature characterizes the

30
30 Stylus Velocity Weighted Velocity
Angular Orientation Stylus Velocity
identity of an individual by how 20 20
Angular Orientation
10 10
unique their features are 0
0
1 2 3
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
User

s = ( D ( p1 || q1), D( p 2 || q2),... D( pn || qn ))
User 3 User 4
User 1 User 2
70
60
Relative Entropy [bits]

50
40
User 5 User 6 User 7
From a geometrical perspective,
30
20

an entropic signature can be 10

0 User 8 User 9 User 10
modeled as a curve in the plane 1 2 3
Biometric Feature
4 5 6

http://www.mcrlab.uottawa.ca MCRLab
Authetication with Analysis of Information Content

Equal Error Rate

Verification EER=4.5%

Performance

Identification

EER=27%

http://www.mcrlab.uottawa.ca MCRLab
Conclusions

User Entropic Signature

X = [ x, y, z , v, F ,τ ,θ ]T

Relative Entropy

X = [ x, y, v, F ,θ ]T
HapticsX = [ Fx,.., Fn]T > TabletsX = [ Fx,.., Fn − i ]

Majority Rule accept 95.5%

+/- 2.5%
X = [ x, y, P ]T
Analysis of
System Security
accept 19.14%
Entropic Signature
X = [σ 2 f x ,σ 2 f y , µ τx , ∑τ y , µ τy ]
+/- 3.5%

http://www.mcrlab.uottawa.ca MCRLab
Future Work

• Incorporating adaptive feedback between

the feature extraction and the feature
selection
• Reducing the size of haptic data
• Combining with other Biometrics
• Embedded in Game-like environment

http://www.mcrlab.uottawa.ca MCRLab
Thank you 

‫متشکرم‬
http://www.mcrlab.uottawa.ca MCRLab
Feedback

• On 1/9/07, Eamonn Keogh <eamonn@cs.ucr.edu> wrote:

Hello, I have read your nice paper [a]. You can make DTW significantly
more accurate for this problem just by constraining the warping, see [b]
or [c] :Professor from University of California USA.
• IEEE-VR Symposium in Virginia USA, March 2006
The main contribution of the paper is that user identification can be
possible in haptic manipulation task using several statistical analyses.
However the result is affected by training existence or nonexistence, so
this can be a disadvantage for haptic systems without trained users, or
those where the user doesn’t frequently interact with the application
• IEEE-HAVE Ottawa, Ontario, October 2005 Reviewer
If the authors could show that the error rates are larger without the
presence of haptics, then it might prove that the haptics are actually
allowing greater behavioral variation, rather than less. For example, what
if the subjects were asked to navigate the same maze with the haptic
feedback turned off?

http://www.mcrlab.uottawa.ca MCRLab
Multiple Point Interaction

x = [t , x, y, z , θ1 , θ 2 , θ 3 , θ 4 , θ 5 ] T

• The state of a haptic system( CyberGrasp) can be characterized by

measurements of virtual hand position (x,y,z), time (t), and angular
orientation of the each finger phalanges (θi)

http://www.mcrlab.uottawa.ca MCRLab
Multiple Point :Accuracy of Measurements

Feature Relative
(dθ1/dt) Dispersion
Average 17.88 %

Standard deviation 285.99 %

RD 22.78 %
Max 42.54 %
Min 22.51 %
(Max – Min) 320.21 %

Causes
•Difficult calibration
•poor measurement
• The average Relative Dispersion (RD)” of a
capabilities set of 20 samples varies about 138.35%.
•Haptic/Graphic rendering This figure immediately dismiss the
 Collision detection possibility of using this data for biometric
purposes

http://www.mcrlab.uottawa.ca MCRLab