Ageing Adaptation for Multimodal Biometrics using ... - IEEE Xplore

5 downloads 8 Views 4MB Size Report
NMIMS University, Mumbai, Maharashtra, India - 400056. Abstract-- Multimodal Biometrics is an emerging domain in claimed identity, identification involveĀ ...

2009 IEEE International Advance Computing Conference (IACC 2009) Patiala, India, 6-7 March 2009

Ageing Adaptation for Multimodal Biometrics using Adaptive Feature Set Update Algorithm H B Kekre, V A Bharadi

NMIMS University, Mumbai, Maharashtra, India - 400056 Abstract-- Multimodal Biometrics is an emerging domain in biometric technology where more than one biometric trait is combined to improve the performance. The biometric system take Face, Fingerprint, Voice, Handwritten Signatures, Retina, Iris, Gait, Palm print, Ear & Hand geometry as common features. Human is identified by correct matching of these features. However, features like face, voice, and signature have low permanence and they change with time. Ageing of human, as well as other psychological & environmental conditions cause gradual change in these features. While enrolling feature set we don't consider this factor. Here we propose a new concept that can be used in designing future multimodal biometrics systems which can adapt to the change in the biometrics features like face, voice, signature, and gait over the time or any other factor without compromising the security. Regression based technique can be used to detect change. This algorithm requires use of at least one biometric

feature which has very low variance or high degree of permanence, like Fingerprint, Iris, Retina etc. This algorithm can address the problem of false rejection caused by sustained change in biometric features due to Ageing or any other factor without the need of re-enrollment of feature set.

I. INTRODUCTION Multimodal Biometrics is a combination of more than one biometric feature for human identification. A biometric is a biological measurement of any human physiological or behavior characteristics that can be used to verify the identity of an individual. Unimodal Biometric systems suffer from several problems like noisy sensor data, non-universality, lack of individuality, non-availability of invariant representations, etc, [1]. These problems are responsible for an increase in error rates and decrease in system reliability for high security needs. Multimodal biometric systems overcome some of the problems associated with unimodal biometric systems by combining the decisions from different biometrics using an effective fusion rule, thus achieving higher accuracy and better performance [2]. A biometric-based authentication system operates in two modes: Enrollment and Authentication. In the enrollment mode, a user's biometric data is acquired using a biometric read and stored in a database. The stored biometric template is labeled with a user identity to facilitate authentication. In the authentication mode, a user's biometric data is once again acquired and the system uses this to either verify the claimed identity of the user or identify who the user is. While verification involves comparing the acquired biometric information with only those templates corresponding to the

978-1-4244-2928-8/09/$25.00 ( 2009 IEEE

claimed identity, identification involve comparing the acquired biometric information against templates corresponding to all users in the database [3]. Here we discuss a different issue for biometric systems. The enrollment of a user for biometric authentication requires acquisition of biometric trait and feature extraction. For example we take photographs for face recognition, fingerprint scan for fingerprint identification, depending on different system different feature set needs to be enrolled. A lot of research work is done on feature extraction & representation. As biometric systems use feature related to human body these features are dependent physical condition of human body which change with age. One more thing is that, not all the features change, features like fingerprint, Iris, retina have very

hegrese san thaedono vary wit tieb face, voice, signature have high variance and they features like ofapermance, change with time. This change is due to the Ageing of human

body, psychological, environmental or other factors. There changes may be sustained or for a short span of time. Current biometric system has user enrollment procedures which take these features from a dataset collected over a very short period of time; this may be a time span of few days to few weeks. It is also not practical to collect a feature set for a

time span of years. We present here a mechanism for enrolling feature set for multimodal biometric system so that it can adapt to change in biometric trait due to Ageing or any other factor. This mechanism is applicable for only multimodal biometric system using more than one biometric trait.

We consider here some standard databases used in

e. dbio database.

A. AR Face Database The AR face [4], [5] contains over 4,000 color face images of 126 people (70menand 56 women), including frontal views of faces with different facial expressions, lighting conditions, and occlusions. T 55 w omn) wee takenrin tof sionsu(seaaed btw images. Twety .wek a) nd eaetincontai or fee iag each session containi c 10 o thes.en12 individuals are selected and used in our experiment. The face portion of each image is manually cropped and then normalized to 50 X 40 pixels. The sample images of one person are shown in Fig. 1. These images vary as follows:


each database to be used in the FVC2002 test, however, is established as 110 fingers, 8 impressions per finger (880 impressions) (Fig. 2). Collecting some additional data gave us a margin in case of collection errors, and also allowed us to systematically choose from the collected impressions those to include in the test databases.

Fig. 1. Example images from the AR Face database.

Another example is Facie database is FERET database [6], using similar data collection mechanism the example is as shown below,

Fig. 3. Example Fingerprints from the FVC 2002

Figure 2. Example images from the FERET Face database.

Organizers of Fingerprint Verification Competition 2002 have designed their own database [7]. Four databases constitute the FVC2002 benchmark. Three different scanners and the SFinGE synthetic generator was used to collect fingerprints. Fig. 3 shows an image for each database. This database mainly concerns hardware variations, different scanners were used for generating database. The details for scanners are as follows. * DBI Optical Identix TouchView II 388374 - 500 dpi a a * DB2 Optical Biometrika FX2000 296 560 - 569 dpi * DB3 Capacitive Precise Biometrics 100 SC 300 300 - 500 dpi * DB4 Synthetic SFinGE v2.51 288 384 - 500 dpiE

A total of ninety students (20 years old on the average) enrolled in the first two years of the Computer Science degree program at the University of Bologna kindly agreed to act as volunteers for providing fingerprints. Each volunteer was invited to present him/herself at the collection place in three distinct sessions, with at least two weeks time separating each session. At the end of the data collection, we had collected for each database a total of 120 fingers and 12 impressions per

finger (1440 impressions) using 30 volunteers. The size of


C. Signature Database We consider a database collected for signature recognition in [8]; this database was collected from 100 people, in more than one session. The database was collected over a time span of average two weeks. A typical record is shown below in Fig. 4. There are various biometrics features databases available over the internet for research purpose. All these databases are collected over a time span of few days to months. These databases do not consider the variations in the feature set due to Ageing. Once the person is enrolled to the system the biometric feature set is static. In the next section we discuss the change in biometric features due to various factors.


X Fig. 4. Example Fingerprints from the Signature database used in [8].

2009 IEEE Internlationlal Advance Computing Conference (IACC 2009)

III. CHANGE IN BIOMETRIC FEATURE DUE TO AGEING As we grow up our physical appearance change as well change occurs in behavioral characteristics. We know that as defined previously biometrics is derived from measurable physical and behavioral characteristics, the feature set of biometrics changes with time. The features that undergo gradual changes are Facial features, Voice, Gait, and Signature. The features like fingerprint, Iris, Retina, DNA have high permanence and they do not change drastically.

A. Facial Features Fig. 5 shows example of such phenomenon. Changes in facial geometry of a person over a time span of 5 & 10 year shown and we can see gradual changes occurring. Fig. 5 (b) shows such changes in a female candidate. If a person is facing a biometric identification system for such a long time we have to design the system so that it can identify these changes and update feature set. B. Signature, Voice & Gate

Similar changes can be observed in signature, voice tone, gait of human being; these features have intra class variations as well as variations due to Ageing and other conditions. IV. NEED FOR ADAPTIVE FEATURE SET From the previous discussion we can conclude tow points





Biometric eatures like face, voice, signature, and

psychological change or other environmental factor. Current feature set enrollment mechanisms are static and the feature set need to be updated as the biometric features change with time. Here we consider extended application of multimodal biometric systems, we can use this multimodal approach in machines having Al and interacting with humans. We can implement the multimodal biometrics system for human identification. We also consider that such human machine interaction will be there for longer time (for years) over which some of the biometric features change gradually, this period and amount of variations is different for different feature and different individual. We now present an algorithm for multimodal biometric System for adaptive feature set updating process. This algorithm divides the biometric feature sets in two categories as follows: 1. Features with low permanence over time: Signature, Voice, Gait, Facial features. 2. Features with high permanence over time: Fingerprints, 2.



Next What?


Palm prin, Iri Reti, (DNA


the biometric feature as 0 eyearsThough n ( CaeififarfF 2EEE Interrlatiorlal Advance are still unique for the individual and

listed above change, they still can be used for identification of the individual. We need to develop a mechanism that can handle this. We must also consider the security aspects, as the forgers should not be allowed to change the feature set& accietludin shldb





Fig. 5. Effect of Agcing on Facial Feature

(b)Chagesinfacal eaureforFemlecanidae 10 ear)

This algorithm is mainly aimed at multimodal biometric based on more than one biometric feature. As the algorithm is adaptive, here we try to design a mechanism that will be able to track the change and update the feature set scrl.Frti ehv odcd h uhniiyo h


,eaturelvs t.oWe definvefolown terms:e ubntct



A. Independent Feature Set (SI) While developing a multimodal biometric system we have to include at least one feature which will remain constant over time. This feature set will be used to confirm the dependent feature set update procedure. Independent feature set includes Fingerprint, Palm Print, Iris, and Retinal Scan.

Fused Srre A




be updated, this include voice, facial features, face, Signature (Static & Dynamic) , Gait etc.

C. Allowed Degree of Variance Allowed degree of variance in the biometric features for which updating is not needed, or the performance metrics are within required limits. For Signature, Voice this variance will be calculated based on a training mechanism as discussed in [9] which is based on Euclidian distance model, and for facial feature the facial regression technique [12] will be used. D. Scale Invariant Feature Transform The SIFT features represent a compact representation of the local gray level structure, invariant to image scaling, translation, and rotation, and partially invariant to illumination changes and affine or 3D projections. SIFT has emerged as a very powerful image descriptor and its employment for face analysis and recognition was systematically investigated in [10] where the matching was performed using three

techniques: (a) Minimum pair distance, (b) Matching eyes and mouth, and (c) Matching on a regular grid.

The present system considers spatial, orientation and keypoint descriptor information of each extracted SIFT point. Thus for example the input to the feature extraction system is the face image and the output is the set of extracted SIFT s = (s 1, S2 .... Sm) where each feature point features si=(x, y, 0, k) consist of the (x, y) spatial location, the local orientation 0 and k is the keydescriptor [10] of size Ix128. Currently we use the SIFT for facial feature and use of this for other biometric features is being studied and is an open issue for research. ....

il I


nw Cnr and SVM



B. Dependent Feature Set (SD) This is a feature set which changes over time and need to


s erfirx









Fig. 6. Fusion network in a Mixture of Expert Architecture (MOE) [1 ].

F. Interval of Update or Needfor Update This is the time after which a biometric feature set is to be updated or a condition indicating need for update. The condition will be decided by classifier of the dependent feature. As shown in Fig. 6, we can see two classifiers, corresponding to an independent biometric feature and a dependent biometric feature. The update will be triggered in following conditions (a) Time limit is reached We set a time period for checking need for update. This time limit depends on the rate of change of dependent biometric trait. At this time we do not have any fixed number for but this can be decided by actual implementation, for a start we can set this to a time period of 1 Month, or even more than that. This is actually depends on Ageing of human being and corresponding change in the biometric feature. (b) Increased FRR in dependent biometric trait Consider case of facial features. This can be due to sudden change in facial geometry, arising from illness, facial makeup, surgery or any other factor. Voices tone my change due to illness or Ageing effect. This will lead to increased vector distance for feature set used while training and input vector may be rejected. This change is to be observed over time and then only update should be triggered. Intelligent module is required to detect this. Considering all these factors we now put a formal description of the steps in Adaptive Feature Set Algorithm. G. Facial Regression In [12] , Zou, Chellappa et al. have used a technique called Image Based Regression (IBR) for Age estimation and Facial

E. Fusion of Multibiometric Features Recognition across Ageing Progression. They proposed a Rather than performing fusion on sensor level or feature . usedecision Bayesian age difference classifier that is built on probabilistic levelhere heredecisionlevelfuo. d n level we use level fusion. We designspcsfa eigenspaces framework. independent classifier for each biometric attribute and fuse the In current cantouse this classifier algorithm decisions of his allows to update the feature in facial featurewedue used the to to change Ageing and cantobedetect of the decisions the classifier. classifier. TThis allowss update the Fure set. We use the scheme as dscussed Udate feature set dnamicall. The classifier need to be mdfe o nerto ncretagrtm network in MOE (mixture-of-expert) architecture. Each vector sequence iS compressed into a local score. The local scores are then fused by a gating network as shown in Fig. 6. us


2009 IEEE Internlationla/Advance Computing Conference (IACC 2009)

VI. ADAPTIVE FEATURE SET UPDATING ALGORITHM This algorithm is working after the decision level of a biometric system; it takes input for different classifiers from the fusion network and monitors the results. The feature vectors are extracted using Scale Invariant Feature Transform (SIFT) (For facial feature). We consider two distinct set of feature vectors (a) Set of dependent feature vectors SD = Isdl, sd2, sd3....sdm where sdi is a feature point as discussed in [9]. With each feature vector we add a counter df corresponding to defaults, i.e. Number of failures while the independent feature vector gives a correct classification.

(b) Set of independentfeature vectors SI = {sil, si2, si2.....sim} For a dependent feature set we define tu as time interval for update in days. We use a classifier based on a model as described in [9], which was initially used for classifying * r 1 1 * * Here we use a variance ofr feature handwritten signatures. vector points and Euclidian distance of input test vector for the median feature vector point for classification. We calculate the variance (cGi), threshold (thi) for distance of input feature vector SDi for the dependent biometric feature. The threshold (thi) is specific for each dependent biometric feature used. This is calculated for the feature vector set used for enrollment of a person and this indicates intra-class variation of the feature. Any feature vector having distance from median feature point less than threshold is accepted and else rejected. This condition is monitored used for decision of update.

Dependent BDomei1c Feaiture Efloikmneflt Senwi & PreprocessingI (Faca, Vbke Sionatura w.)

Feature V&Ktar Extrant n (Scale Invariant Feature

Nns T


Daabs li E nrblasn 1TIm|l

One Classifier which belongs to independent feature plays an important role. Here we have multibiometric system and fusion algorithms working for combining the decisions from these classifiers are used for authenticity of user and if the classifier giver input feature vector SI as correct then only the dependent feature vector SD can be updated if update condition is triggered. Here we consider a case of multimodal biometrics system having two classifiers where one belongs to Fingerprint Verification (Operating on SI) and other belongs to Facial Feature Classifier (Operating on SD). A AF A tVUAlgorithme 1. Reset the time interval ti=O, default counter for dependent feature df=O; 2. Start the multimodal biometric system. Read the input test vector ST={SDT,SIT} for user Un Where ST= Test feature vector set SDT = Test Dependent Feature vector (Dependent Feature vector Subset) SIT = Test Independent Feature vector Feature vector Subset) (I(~~~~~~~~~~~Independent pe) 3. If Adaptive Gating Network Classifier REJECTS the test vector ST {SIT+SDT} then GO TO step 4. Else go to step 2 (Next Test). 4. If Independent Feature Classifier REJECTS the test feature vector (SIT) then go to Step 2 else GO TO Step 5. (REJECTED TEST VECTOR). NO NEED TO UPDATE. 5. If Dependent Feature Vector Classifier REJECTS the

feature vector ST{SDT} (While dependent feature vector is ACCEPTED) Mark as default condition df=df+1 for user Un.

Classer for Dependent u re Set


~~~~~~~~~~~~~~~~~~~~~~~nl Deiso

Central Nature


tEtWFig. 7. Multmoda Bimeri Bl3ometn. Feaure ~I ddpendntEiriniIk;ti Feature Vedctr EAracioIn

Senso & PreprocssiAng

(Fingprnt, Iris. Reinaet.)

Database EnroIIment

1vat Fetue

Vector~Update -4,

k1 ID1 Ik






Syte usn Adptv Fetr Veto Upat mechanism.iit

Classifier idr I i1 mentidri Biometntr Fealure Seta

209IEEItentiona Advnc CoptpCinXg Cofrec (IC 209


6. If default count df > permitted defaults (DP) for user Un, INVOKE Feature vector UPDATE MODULE. Reset Default counter df =0, for user Un. GO TO step 2 (This (df=O) will avoid unnecessary repeated update). 7. If Age difference classifier detects valid change in biometric feature, INVOKE Feature vector UPDATE MODULE. Reset Default counter df =0, for user Un. GO TO step 2. 8. If time interval to update ti > permitted limit (TP) of user Un then INVOKE the Feature Vector UPDATE MODULE. Reset ti=0,df=0 for user Un. GO TO step 2.

B. Feature Vector Update Module This is very important module, this works on the same lines as the enrollment module for each of the classifier. We operate the module on the FIFO basis, oldest feature vector is deleted and the latest Feature vector will be added. The new feature vector will be added to the set and we calculate the variance (oi), threshold (th.) for distance of input feature vector SDi for

the dependent biometric feature with the included feature vector. This forms the updated feature vector set (SDi) for user Un. The steps are as follows. 1. Prompt user Un for UPDATE of Feature Vector stating the reason 2. If allowed by user, read again the feature vector for corresponding dependent feature vector. Find SIFT [10] as discussed earlier. Tag the feature vector with current date & time 3. Delete the oldest feature vector from the set SD = {Sdl, sd2.. ..Sdn}. 4. Using the training mechanism in [9] find out variance (i)

threshold (thi) for the updated set, Update the corresponding record. 5. Prompt the user of the Completion of Update Process.

C. Security in Updating Process As the feature vector update is triggered only when the Independent Feature Vector successfully ACCEPTED, this adds security against false updating of feature set. We take fingerprint, Palm print, Retina, and Iris as Independent feature vector, there biometrics features have high CCR (Correct Classification Rate) and very difficult to forge as compared to the dependent features. As no other entry point for triggering the updating (INVOKE UPDATE MODULE Event) process, we can say that the update process is secure

Currently unimodal biometric systems are ubiquitous and multimodal biometric systems are having low penetration owing to high cost and complex structure [13]. But where we need high accuracy and sustainability, this algorithm can surely help to improve performance of the system. The application of this algorithm is in all multimodal biometric systems, with very large time span for operation over a specific population like Schools, Offices, Hospitals, Military, Research Laboratory Access control equipments. Another application is development of Machines with Al and high degree of human interaction. Implementation of this algorithm does not require change in

any of the hardware (Sensors). Change in coding architecture

is required and this can be integrated with existing work. Current multimodal biometric system designers should consider this algorithm for implementation. This algorithm is proposed for improving performance of multimodal biometrics systems and imparts adaptability towards change in biometric features due to Ageing by making these systems aware of gradual changes in human biometric trait over the time. Performance of such system should be tested for longer time over a real time application scenario. This algorithm is not final and may need revisions as many of the parameters are still unknown, like permitted number of defaults (DP) and time interval to update (TP) & design of age difference classifier need further research. Better feature vector extraction, training & enrollment mechanisms can be combined with this algorithm for improving performance. REFERENCES [] K. Nandkumar, "Integration of Multiple Cues in biometric Systems", PHD

Thesis, Michigan State University, 2005.

[2] Teddy Ko, "Multimodal Biometric Identification for Large User Population Using Fingerprint, Face and Iris Recognition", Proceedings of 34th Applied Imagery and Pattern Recognition Workshop (AIPR), 2005.

[3] Recognition A. Ross andLetter A.K. Jain, "Information 2003. in Biometrics", Pattern 2125,Fusion 24, pp.2115[4] A.M. Martinez and R.

Benavente, "The AR Face Database," CVC

Technical Report #24, June 1998. [5] A.M. Martinez and R. Benavente, "The AR Face Database", 2006.

[6] P.J. Phillips, "The Facial Recognition Technology (FERET) Database,", 2006.

[7] D. Maio, D. Maltoni, R. Cappelli, J.L. Wayman, A.K. Jain, "FVC2002: Second Fingerprint Verification Competition", 16th ICPR [8] HVariance B Kekre,Analysis V A Bharadi, A Ambardekar, "Signature Recognition by Pixel Using Multiple Morphological Dilations", International Journal of Information Retrieval (IJIR), International Sciences Press, Vol. 1 no. 1 June 2008, pp 5-9, ISSN: 0974-6285


The Adaptive feature vector algorithm is proposed for development of complex multimodal biometrics systems with large span. operating The authors are in process of large operating time development of classifier and multimodal biometric system. At this moment we present the algorithm conceptually. This algorithm is very crucial as we will have more and more multimodal biometric systems coming in human interaction.

Fig.7. Showsarchitectue for proposed biometri boerc forpropsedmultimodal ach1tctur ShwsAdaptive Flg.. Vector Update algorithm. Feature system with


[9] B. Majhi, Y S Reddy, D Prasanna Babu, "Novel Features for Off-line Signature Verification", International Journal of Computers,

Communications & Control Vol. I, 2006 [10] M. Bicego, A. Lagorio, E. Grosso and M. Tistarelli, "On the use of SIFT features2006. for face authentication", Proceedings of CVPR Workshop, New [11] s. Y. Kung, Man-Wai Mak, "On Consistent Fusion Of Multimodal Biometrics", ICASSP 2006, June 2006. [12] S K Zhou, R. Chellappa, W. Zhao, "Unconstrained Face Recognition", [1]Springer Science, 2006, ISBN-1O: 0-387-26407-8, pp 111-127 A. K. Jamn, K. Nandakumar A. -Ross and,387-22296-7, "Handbook 2006. of ISBN 978-0Multibiometrics", Springer, and


2009 IEEE Internlationlal Advance Computing Conference (IACC 2009)

Suggest Documents