Anda di halaman 1dari 8

Data fusion in biometrics based on Empirical Mode Decomposition

Introduction
Data and information fusion is used in many engineering applications, including biometrics, biomedical engineering and communications. This research aims to investigate the benefits of real time adaptive algorithms for data fusion in biomedical applications, such as in biometrics and the modelling of human behaviour. This will be based on a rigorous study of input variable selection and the relevance of the features in this context. Experimental tests will be conducted in our new Smart Environments Lab and will involve recording of real world situations from multiple heterogeneous sensors. Results from experimental measurements will be used to develop improved analytical models for knowledge extraction based on the estimates of fundamental signal properties (nonlinearity, sparsity, smootheness) as the model features.

Data fusion
Data fusion is a framework to combine data from multiple sensors in order to achieve more efficient and potentially more accurate inferences than that achieved by means of a single source. This concept is inspired by living organisms who have the capability to use multiple senses to learn about the environment. The brain then fuses all this available information to perform a decision task [1]. In fields as diverse as recognition, identification, tracking and decision making, complementary signals coming from multiple sources are processed within data fusion framework. Fusion processes are often categorized as low, intermediate or high level fusion depending on the processing stage at which fusion takes place [2]. Low level fusion, also called data fusion, combines several sources of raw data to produce new raw data that is expected to be more informative and synthetic than the inputs. Intermediate level fusion, also called feature level fusion, combines various relevant features
extracted from signals coming from different sources and/or extracted by different methods such as Fast Fourier Transform (FFT), Discrete Cosine Transform (DCT), wavelet, Delay Vector Variance (DVV) [3] and Empirical Mode Decomposition (EMD) [4]. High level, also called decision fusion combines decisions coming from several

experts. Methods of decision fusion include voting methods, statistical methods, fuzzy logic based methods, etc. In practice, these three levels can be applied in any combination, for example, Data in-Data out, Data in-Feature out, Feature in-Feature out, Feature in-Decision out, Decision in-Decision out [2].

Fusion in biometrics
Biometrics refers to the automatic identification (or verification) of an individual (or a claimed identity) by using certain physiological (such as fingerprints, iris, face etc.) or behavioural (gait, typing rhythm, mouse gestures) traits associated with the person [5]. Compared to the traditional knowledge-based security (like code or password) or possession-based security (like security tag or ID card), the biometrics are clearly at the forefront as they can not be easily stolen or shared. A simple biometric system has four important components (Fig. 1). Sensor module acquires the biometric data of an individual which is then processed to extract feature values in the Feature extraction module. In Matching module, the extracted features are compared against those in the template by generating a matching score. And based on this, the claimed identity is either accepted or rejected in the Decision-making module [5].

Fig. 1. A simple biometric system The performance of a biometric system is largely affected by noisy sensor data and the restricted degrees of freedom of the features extracted from the sensed signal. This can be addressed by using multimodal biometric systems [6] where the recognition accuracy of classifiers is improved by fusing the complementary information of the individual classifier [6, 7]. Moreover, multibiometric systems provide anti-spoofing measures; alleviate the problem of non-universality of some biometric traits (e.g., some individuals fingerprints can not be extracted reliably due to the poor quality of the ridges) as well as unavailability of certain biometric data (e.g., visual signals such as face, ear, etc. might be occluded in surveillance videos). As a result, a large body of active work is aimed to combine various biometric such as face, fingerprint and 3

hand geometry [5], face, fingerprint and speech [8], face and iris [9], face and ear [10], face and speech [11] and face and gait [12].

Fusing face and gait


Approaches identifying face, a physical biometric, can be broadly divided into two categories: 1) Feature-based methods [13, 14] which process the input image to extract distinctive facial features (such as the eyes, mouth, nose, etc.) and compute the geometric relationships among them as feature vector. Standard statistical pattern recognition techniques are employed thereafter. 2) Appearance-based (or holistic) methods [15, 16] which try to recognize faces using global representations, i.e., descriptions based on the entire image rather than on local features of the face [12]. On the other hand, gait is a behavioral trait. The human walking process is similar for all humans despite variable walking styles. A typical sequence of stances in a gait cycle (Fig 2) is composed of the following four main walking stances [17]: right double support (both legs touch the ground, right leg in front), right midstance (legs are closest together, right leg touches the ground), left double support, and left midstance. A gait cycle is defined as the interval between two consecutive left/right midstances. The time interval in which a gait cycle is carried out is called the gait period, whereas the walking frequency is termed the fundamental gait frequency.

Midstance

Double Support

Midstance

Double Support

Midstance

Fig. 2. Several stances during a gait cycle [18]. Approaches identifying people from gait can broadly be partitioned into two groups [19]: 1) Model-based methods which apply a model of either the persons shape or motion to obtain static (i.e., absolute distances between certain body landmarks, such as the height, limb lengths, shoulder width, torso length, etc.) as well as dynamic gait features (e.g., swings of legs and arms, stride length, walking speed, and kinematics of joint angles) 2) Holistic techniques which represent the gait by the statistics of the spatiotemporal patterns generated by the walking person in an image sequence [12]. 4

It is noteworthy to mention that some approaches extract gait features from modalities other than video such as in [20] where floor sensors are utilized to extract stride parameters as well as time on toe to time on heel ratios. As the integration of the face with gait for identity verification is a relatively unexplored area, our research will focus on designing novel strategies for the fusion of face and gait. Besides some specific reasons for considering the fusion of the face and gait biometrics, in particular, are as follows: the face is a short-range biometric, whereas gait is a medium to long-range biometric. As a result, these two biometric traits together would arguably make the system more robust to distance variations with respect to the camera. Also, both face and gait are visual signals; both can be extracted from the same modality (i.e., image sequences of people) which precludes the need for separate or specialized equipment. Furthermore, some conditions such as large variations in illumination and facial expressions sharply degrade the performance of face recognition systems but affect gait to a much lesser extent or not at all. Similarly, some conditions such as clothing, footwear, and load adversely affect the accuracy of gait recognition but do not influence the performance of face recognition systems [12]. Hence, fusing those two complementary signals would certainly improve the recognition accuracy.

Proposed work
Though the development of the integration of face and gait is still in its nascent stages, most of the proposed systems so far fuses these two traits at one of the three levels: 1) the feature level, where feature vectors of the individual biometrics are concatenated to construct a new feature vector representing the subjects identity in a higher-dimensional and apparently more discriminating hyperspace. 2) the matching score level, where the similarity scores of the individual biometrics are first transformed (using transformations such as linear, exponential, logarithmic, etc.) to make sure that they map to a common range and then combined using some combination rule (e.g., SUM, PRODUCT, MAX, etc.) to get a new similarity score. 3) the decision level, where each individual classifier takes its own decision regarding the identity of the unknown person and some rule (e.g., majority voting, rank rule, 5

etc.) is applied to make the final decision. Hierarchical systems, in which a weak classifier is used to prune off the most unlikely candidates and the remaining candidates are passed to a more powerful classifier which makes the final decision, may be assigned to this category [5, 12]. In our proposed work, we will try to devise some novel strategies to fuse these two traits in each level and compare the result in terms of recognition accuracy. In the first phase of our work, we will explore feature-level fusion. For that cause, we will use a feature extraction technique based on recently introduced empirical mode decomposition algorithm. The motivation behind this is as recent research shows that gait is both nonlinear and non-stationary in nature [21]. The face feature vector will be constructed using one of the existing methods.

Empirical mode decomposition


Empirical Mode Decomposition (EMD) is a versatile data-dependent decomposition of a time series into waveforms modulated in both amplitude and frequency [4]. The basis of the expansion, called the Intrinsic Mode Functions (IMFs), is generated in a direct, a posteriori, and adaptive way. Each signal x(t) can be decomposed as follows:
x (t ) = d i + rn
i =1 n

(1)

where di is an IMF and rn is the mean trend of x(t) or residue. For one IMF di(t) in equation (1), its Hilbert transform is defined as
H [ d i (t )] =
d i (t / ) / 1 dt t t /

(2)

With the above definition, an analytic signal can be formed as

z i (t ) = d i (t ) + jH [d i (t )] = ai (t )e ji (t )
From equation (3), the instantaneous frequency (IF) is obtained with
IFi (t ) = 1 di (t ) 2 dt

(3)

(4)

A range of features will then be extracted from both the IMFi (i=1, 2, 3) and the analytic signals to form a robust gait representation. Based on some feature reduction techniques, useful features will be selected from them. Thus, the resulting vectors from gait and face will be concatenated to form combined feature vector. At last, the 6

recognition or classification will be achieved using either any template-based approach or stochastic approach. Besides, in our newly built smart environment lab using the Tekscans (www.tekscan.com) HR Mat which is a high-resolution floor mat system for capturing barefoot plantar pressure and assessing foot function, the gait will be analyzed and several gait features will be extracted. And fusion with existing video extracted gait information will also be investigated.

Conclusion
By processing complementary information coming from different sources, data fusion framework achieves improved accuracies and more specific inferences that could not be achieved by the use of only a single source. Our proposed work will investigate the problem of data fusion in biometric verification and formulate novel strategies for fusing different biometric modalities based on fundamental signal properties.

References
[1] D. P. Mandic, D. Obradovic, A. Kuh, T. Adali, U. Trutschell, M. Golz, P. de Wilde, J. Barria, A. Constantinides, and J. Chambers, "Data Fusion in Modern Engineering Applications: An Overview," in Proceedings of ICANN '05, 2005, pp.715 - 721. [2] B.V. Dasarathy, Sensor fusion potential exploitation Innovative architectures and illustrative applications, in Proceedings of the IEEE, 1997, vol. 85, 2438. [3] T. Gautama, D. P. Mandic, and M. M. Van Hulle, "A Novel Method for Determining the Nature of Time Series," IEEE Transactions on Biomedical Engineering, vol. 51, no. 5, pp. 728-736, 2004. [4] N. E. Huang, Z. Shen, S. R. Long, M. L.Wu, H. H. Shih, Q. Zheng, N.C. Yen, C. C. Tung, and H. H. Liu, The empirical mode decomposition and Hilbert spectrum for nonlinear and non-stationary time series analysis, Proc. R. Soc. London A., vol. 454, pp. 903995, 1998. [5] A. Ross and A. K. Jain, "Information fusion in biometrics," Pattern Recognition Letters, vol. 24, pp. 21152125, 2003. [6] L. Hong, A. Jain, and S. Pankanti, "Can multibiometrics improve performance? in IEEE Workshop Identification of Advanced Tech., 1999, pp. 5964.

[7] B. Schiele, "How many classifiers do I need?" in Int'l Conf. Pattern Recognition, 2002, vol. 2, pp. 176179. [8] A. K. Jain, L. Hong, and Y. Kulkarni, "A multimodal biometric system using fingerprints, face and speech," in 2nd Intl Conf. AVBPA, 1999, pp. 182187. [9] Y. Wang, T. Tan, and A. K. Jain, "Combining face and iris biometrics for identity verification," in Int'l Conf. AVBPA, 2003, pp. 805813. [10] K. Chang et al., "Comparison and Combination of Ear and Face Images in Appearance-Based Biometrics," IEEE Trans. PAMI, vol. 25, pp. 1160-1165, 2003. [11] J. Kittler et al., "On combining classifiers," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, pp. 226-239, 1998. [12] R. Jafri, and H. R. Arabnia, Fusion of Face and Gait for Automatic Human Recognition, in 5th Intl Conf. ITNG, 2008, pp. 167 173. [13] I. J. Cox, J. Ghosn, and P. N. Yianilos, "Feature-based face recognition using mixture-distance," in Proc. IEEE Conf. CVPR, 1996, pp. 209-216. [14] L. Wiskott et al., "Face Recognition by Elastic Bunch Graph Matching," IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 19, pp. 775-779, 1997. [15] M. Turk and A. Pentland, "Eigenfaces For Recognition," J. Cognitive Neuroscience, vol. 3, pp. 71-86, 1991. [16] P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, "Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, pp. 711-720, 1997. [17] R. T. Collins, R. Gross, and J. Shi, Silhouette-based human identification from body shape and gait, in IEEE Conf. Automatic Face and Gesture Recognition, Washington, DC, May 2002, pp. 351356. [18] N. V. Boulgouris, D. Hatzinakos, and K. N. Plataniotis, Gait Recognition: A challenging signal processing technology for biometric identification, IEEE Signal Processing Magazine, vol. 22, no. 6, pp. 78-90, 2005. [19] C. BenAbdelkader, R. Cutler, and L. Davis, "Person Identification using Automatic Height and Stride Estimation," in IEEE Int'l Conf. Pattern Recognition, 2002, vol. 4, pp. 377 380. [20] L. Middleton et al., "A Floor Sensor System for Gait Recognition," in 4th IEEE Workshop AutoID'05, 2005, pp. 171-176. [21] C. Peng, J. Hausdorff, and A. Goldberger, "Fractal mechanisms in neural control: Human heartbeat and gait dynamics in health and disease." In J. Walleczek, editor, Self-Organized Biological Dynamics and Nonlinear Control. Cambridge University Press, 2000. 8

Anda mungkin juga menyukai