Handwritten Signature Recognition in Mobile Scenarios - IEEE Xplore

16 downloads 266 Views 441KB Size Report
device each; 20 skilled forgeries signatures per user were used also. The devices used were mobile phones, tablets, laptops and two specific devices for signing ...
Handwritten Signature Recognition in Mobile Scenarios: Performance Evaluation Ramon Blanco-Gonzalo, Oscar Miguel-Hurtado, Aitor Mendaza-Ormaza, Raul Sanchez-Reillo University Carlos III of Madrid University Group for Identification Technologies (GUTI) Avda. Universidad, 30; 28911 Leganes (Madrid), SPAIN {rbgonzal, omiguel, amendaza, rsreillo}@ing.uc3m.es

This work was done in order to evaluate the performance achieved in the migration of a biometric modality to mobile scenarios. The biometric modality chosen has been a DTWbased handwritten signature recognition solution [1], and it has been applied to different platforms and devices and combinations of them.

Abstract— Following the idea of improving our previous work on dynamic handwritten signature recognition on portable devices, a performance evaluation in a mobile scenario was done. A database with 11 users and 8 mobile devices (using stylus and finger) has been collected in order to study different parameters such as screen size, operative system and the interoperability between the devices. The evaluation was divided by 3 sessions of 20 signatures per device each; 20 skilled forgeries signatures per user were used also. The devices used were mobile phones, tablets, laptops and two specific devices for signing. The algorithm used to assess the signatures was a DTW-based signature recognition algorithm.

The information acquired is detailed in this paper, as well as the kind of results obtained. The experiments done consist of applying the proposed algorithm to the signatures gathered in all the devices in order to compare the results obtained in each one. The same process was done using skilled signatures. Finally, the interoperability among devices was studied exchanging their templates. This paper is organized as follows: in section II the database obtained and the devices features are explained as well as the methodology carried out for obtaining the samples. Next, in section III, the algorithm characteristics are described. The experiments done are explained in section IV and the final results are included in section V. Finally, in section VI, the conclusions obtained from the study are presented.

Keywords-component: On-line handwritten signature; biometrics; evaluation; mobile devices; screen size; interoperability I.

INTRODUCTION

Biometrics can be considered as a technology that is entering maturity. It is also applicable to those applications and services where the user is currently being authenticated by using passwords or tokens. In addition, current trends in service development show the need of deploying services in mobile devices, such as smart phones and tablets. Therefore migrating biometrics to such platforms is one of the hot topics in R&D nowadays. But migrating to mobile devices brings new challenges that have to be covered. These new challenges are, among others:

II.

As it has been presented in the introduction, the devices used in the evaluation and the database capture methodology are explained in this section, that is divided by two parts. First, a description of the devices used is provided and then the structure of the database and the methodology of capture are broke down. The description of the devices included in the evaluation is basically the main features that are interesting for research, such as dimensions or technology.

a) Adapting the device to acquire the selected biometric modality.

A. Devices Seven different devices were used in the evaluation. These are 2 Wacom tablets, 1 tactile laptop, 1 mobile phone and 3 tablets. Some of them have a stylus and others require just the finger for signing.

b) Defining the application architecture to better fit the scenario (e.g. local authentication vs. remote authentication). c) Fine tuning the biometric algorithm for improving performance considering the execution platform and the acquisition properties.

Being the most widely used devices in handwritten signature, Wacom tablets are used here as a reference. As these tablets were designed specifically for signing, their use is ergonomic and comfortable for the user, as well as they provide a large variety of parameters of the signature, like pressure and point coordinates and all of them with a large resolution. Specifically, the devices used in this work are Wacom STU500 [2] and Wacom Intuos 4 [3], where this last one does not provide visual feedback of the signature made. The software

d) Evaluating the impact on performance of the implementation of the solution in a variety of platforms. e) Evaluating the impact on performance of the different ways and situations of using mobile devices. f) Evaluating the usability of the new applications and redesigning them according to the results obtained.

978-1-4673-2451-9/12/$31.00 ©2012 IEEE

DEVICES AND DATABASE

174

for controlling Wacom devices was developed with Microsoft Visual Studio 3.5 [4].

screen, so it is measurable and the concrete point touched can be localized. Therefore, a finger touch can be detected in a capacitive screen because the human body acts as a conductor. In this study 3 of the devices are capacitive and only one of them has its own special stylus that conducts electricity (Note).

One tactile laptop (Asus EeePC T101MT [5]) is included in the evaluation also. As with Wacom devices, the program for signing was developed with Microsoft Visual Studio.

A resistive touch screen has several layers and two of them are separated by a gap. One of the two layers has voltage passing through it and the other is able to sense this voltage. So, when something makes pressure over the upper layer it involves a contact with the other layer that can be detected. A resistive screen does not need a conductor for detecting a touch, so the resistive device used in the evaluation (Asus EeePC) has a stylus in order to be more comfortable for signing. Furthermore, the Asus EeePC allows multi-touch, which is a non-common feature in a resistive device, but this is not an influence factor in the signature process.

One specific mobile phone (Samsung Galaxy Note [6]) has been used to sign with both, the finger and its own stylus. Note’s software was developed with Android SDK [7]. The 3 tablets used in the evaluation were iPad [8], Samsung Galaxy Tab [9] and Blackberry Playbook [10]. In all these three cases, the signature was performed using the finger. The finger used is up to the user to decide. iPad’s software was developed in Objective-C with the Apple SDK [11]. The Galaxy Tab application was the same as the Note’s and the Playbook software was developed with the SDK from Blackberry [12] TABLE I. Screen Device Size Wacom 5 in STU-500 Wacom 10,81 in Intuos 4 Asus EeePC 10,1 in MT101 Samsung Galaxy 5,3 in Note iPad 9,7 in Samsung Galaxy Tab Playbook

3) Visual feedback The visual feedback is a parameter that can affect the final signature due to usability. It is common for the user to see his/her sign, but it not so common signing without seeing how the signature is being performed. The objective is to measure how much it affects to the algorithm throughput and if a device without visual feedback is interoperable with the other devices. All the devices except the Wacom Intuos 4 provide visual feedback.

DEVICES FEATURES Visual Technology Modality feedback Capacitive

YES

Stylus

Capacitive

NO

Stylus

Resistive

YES

Stylus

Capacitive

YES

Stylus/Finger

Capacitive

YES

Finger

7 in

Capacitive

YES

Finger

7 in

Capacitive

YES

Finger

4) Modality As mentioned above, one of the trending topics in the recognition by handwritten signature is to check if signatures made by using stylus and those made by using just the finger, can be considered as the same biometric modality. Half of the devices are used with stylus and the other half with the finger, so it is feasible to make a comparative study. In the results section the interoperability and the error rates are shown. B. Database The database collected is composed by signatures obtained from 8 devices. There were 11 users in the evaluation, 2 women and 9 men, 1 left-handed and 10 right-handed. All of them were engineers between 24 and 39 years old and totally familiarized with the technology. Furthermore, they used their real signatures. Regarding the scenario, for providing the signatures the users were sitting on a chair and the devices were placed on a table, so that the users could sign comfortably.

The main device features that were studied are the screen size, the technology, the "modality" (stylus/finger) and the visual feedback, as the most influence factors. It is important to note that in the very beginning of this study, the hypothesis of being studying two different biometric modalities is stated, being one of those the traditional signature with stylus, and the other one the handwritten signature with a finger. As it will be seen later on, this hypothesis will be either confirmed or discarded according to the results obtained.

The skilled forgeries were the same users between themselves having advanced knowledge about the signature to forge. In fact, the skilled forgeries were produced after a learning process in which the forger (each user) observed the user to be impersonated signing several times. So, the impostor has repeatedly witnessed of the claimed user’s signature.

1) Screen Size Each device has a different screen size that is considered as a differentiate factor due to the different signature that the user provides depending on the space that she has for signing. According to the screen sizes it is feasible to divide the devices by three groups, small size (5"), medium size (7") and big size (10"). The screen sizes and other features are summarized in Table I. It is important to remark that the screen size refers to the length of the diagonal of the screen.

The evaluation was divided by 3 sessions in which the user had to sign 20 times in each device. It supposes 60 signatures made by 11 users in 8 devices. Additionally, each user had to provide 10 forged signatures of 2 different users in each device. The aggregated database is composed of 7040 signatures. The order of the devices was randomized and the user had a little break between each device in order to avoid the effects of habituation. At least two weeks separation was scheduled between sessions for avoiding tiredness effects also.

2) Technology The most well-known mobile touch screen technologies nowadays are the capacitive ones and the resistive ones. The capacitive screens are composed by an insulator and a conductor. When the surface of the capacitive screen is touched by another conductor it affects the electrostatic field of the 175

III.

experiments it is pretended to assess the performance of the algorithm with the data captured by each device. The importance of the screen size, the technology and the visual feedback is measured too. Many comparisons between devices are needed in order to obtain further conclusions about the relevant parameters as it is shown in Table II.

ALGORITHM

In this section a brief description of the handwritten signature recognition algorithm chosen is provided. This algorithm [13] is a DTW-based one and is made for on-line signature recognition. In previous works [14] this algorithm was modified for being adapted to mobile devices. The algorithm error rates in previous experiments with portable devices were EER = 1.8% for random forgeries and EER= 7.6% for skilled forgeries. A detailed description of the algorithm and the results obtained can be accessed via the referenced papers.

TABLE II. Screen size Intuos-Asus-iPad vs. Playbook-Tab vs. STU-Note

The main point of this paper is to continue previous works, by testing the algorithm with more advanced and more common mobile devices. Also, another important topic of discussion is to analyze the interoperability among them. By the end of the paper, it will be tested out if the algorithm’s error rates are acceptable with these new devices or if it is necessary to modify either the algorithm, or the acquisition process.

B. Anti-spoofing Resistance In this case, the algorithm also compares the template with the advanced skilled forgeries generated. As it was said before, it is expected that the error rates are bigger in these experiments due to the training process of the forgers. A comparison among all devices is done.

For compatibility, the parameters used as input data to the algorithm were only time, X and Y signals. This restriction comes from the fact that some devices which do not capture pressure, so such signal has been discarded in the processing of the information coming from any of the devices.

C. Device Interoperability For the interoperability experiments first one device is used for obtaining the user’s template and then this template is compared with the signatures acquired with other devices. The process is repeated for each device in order to test which devices are more compatible among each other.

Each device returns one file per signature containing the X, Y coordinates and time data in ISO/IEC 19794-7 format [15]. This file is stored in a database for later comparisons. The enrolment process has been adapted to a typical scenario in the real world. In order to avoid user lack of comfort, only 3 signatures are considered for the enrolment. The 3 signatures used for enrolment have been the first 3 ones, as to copy what the enrolment process in a deployed application may look like. These two constraints are important pieces of information when comparing the error rates obtained to some of the ones found in the literature, where at least 5 samples are used, and they are chosen from the whole set of samples in the database.

V.

RESULTS

In this section, the results of all the experiments done are shown. For carrying out the evaluation, the algorithm output was designed for obtaining the EER. Next, the different rates are compared by sections and tables. The results are divided by three parts as it was planned before: Performance, Antispoofing resistance and Device Interoperability.

After the enrolment phase, the verification process has been done with the rest of the 57 signatures from each of the users and each of the devices. With such a method, and considering a system configured to work with transactions of 1-single attempt, the False Rejection Rates (FRR) and False Acceptance Rates (FAR) are obtained. As a preliminary measure of the results achieved the Equal Error Rate (ERR) will be used, although it does not provide all the information about the algorithm performance but it can express a relationship between FAR and FRR and the accuracy of the biometric system.

A. Performance The results of the experiments with genuine signatures are shown in this part. In addition to the studies carried on in this section, a secondary objective is to compare these results with the ones of the section dealing with Anti-spoofing resistance. 1) Visual feedback results The first experiments were done in the line of discovering if the visual feedback was a really influent factor for the users’ signature. The devices chosen for the comparison were both Wacom devices because both have similar features: both devices return the same data (coordinates and pressure) and they have a similar sampling frequency (one point each 10 ms approximately). But one of them has no visual feedback (i.e. Intuos 4). The EER obtained reveals a huge difference between then as can be seen in Table III.

In the case of the skilled forgeries error rates, the enrolment is done in the same way, and the 20 impostor signatures of each user are added to the number of signatures to be used for comparing with such an user. IV.

Experiments executed Visual Technology Modality feedback Intuos-Asus-Note All STU stylus-STU vs. vs. vs. Intuos Asus Playbook.TabiPad-Note finger

EXPERIMENTS

After finishing the database various experiments for calculating the error rates using the DTW algorithm were done. Those experiments are classified in three groups: Performance (a.k.a. random forgeries), Anti-spoofing resistance (a.k.a. skilled forgeries) and Device Interoperability.

TABLE III. Visual feedback EERs Device Visual feedback EER STU YES 1.27% Intuos NO 22.90%

A. Performance The algorithm receives signatures from other users as to determine the discriminative power achieved. With these 176

In Figure 1 and Figure 2 graphics with the ROC curves of each device are shown divided by modality, as a visual characterization of the trade-off between the FAR and the FRR.

It shows that it is very important for the user to receive a visual feedback during the signing process. The huge difference between the EERs may be due to multiple factors. An exhaustive revision of the algorithm is proposed as a future working line to clarify the results. As the results obtained with the Intuos device were much worse than expected, this device will not be used for the rest of the experiments.

Therefore, in few words it cannot be stated that both are two independent modalities, but that they can be considered as the same biometric modality with different technologies for acquiring data.

2) Screen size results The following analysis was focused on comparing the results according to the available space for signing. Seven devices were divided in three groups in order to categorize the screen size. TABLE IV.

Screen size EERs

Devices

Size

EER

Asus and iPad

Big

3.48% and 0.47%

Playbook and Tab STU and Note (finger/stylus)

Medium

1.39% and 2.38% 1.27% and 0.29%/0.17%

Small

The results are not totally enlightening due to the variability among devices rates of the same size as it is shown in the Table IV. But somehow it can be said that those devices with small size are considered as the best ones.

Figure 1. ROC - Stylus devices

3) Technology results The resistive touch screen device (Asus EeePC) is compared with the capacitive screen devices in this section. Selecting the capacitive device with the worst EER (Tab), the resistive device has obtained even worse rates. The Table V shows that there is not so much difference between them, but the resistive device has a slightly higher EER. TABLE V.

Technology EERs

Devices

Technology

EER

Asus

Resistive

3.48%

Tab

Capacitive

2.38%

4) Modality results Figure 2. ROC – Finger devices

Attending to the modality, four devices of each type (stylus and finger) were tested. There exists so much variability among the results of those devices from the same modality. So that, there are good error rate’s devices and bad error rate’s devices with stylus. The same thing happens when signing with the finger. This is shown in Table VI.

B. Anti-spoofing Resistance The same experiments using skilled forgeries for obtaining error rates were done. In Table VII it is shown a comparison between the results obtained using only genuine signatures and those adding skilled forgeries. In more detail, Figure 3 illustrates the results for the stylus-based devices. As expected, significant differences between their EERs have been found. Some cases present even more than 15% or an increase in EER. Also, there is no significant difference between the behavior of the stylus-based devices and the one of the finger-based devices.

In short, with the results obtained, the best EER is provided by the Samsung Galaxy Note, which results are below 0.3% (0.17% with stylus and 0.29% with the finger). TABLE VI.

Modality EERs

Devices

Modality

EER

Note stylus – STU - Asus

Stylus

0.17% - 1.27% - 3.48%

Note finger – iPad – Playbook - Tab

Finger

0.29% - 0.47% - 1.39% 2.38%

TABLE VII. Device STU

177

Random forgeries vs. Skilled forgeries Random forgeries Skilled forgeries (EER) (EER) 1.27% 11.10%

Asus Note stylus Note finger iPad Playbook Tab

3.48% 0.17% 0.29% 0.47% 1.39% 2.38%

TABLE IX.

19% 10.10% 10.10% 7.99% 9.25% 17.18%

Note interoperability

Device

Note stylus template EER

Note finger template EER

Note stylus

0.17%

0.93%

Note finger

0.34%

0.29%

VI.

CONCLUSIONS AND FUTURE WORK

Multiple outcomes can be obtained from this ongoing work. As results were divided in three sections, the conclusions follow the same organization: Performance, Antispoofing Resistance and Device Interoperability. A. Performance Attending to the EERs obtained with the Intuos and the STU the differences between them are considerable, so the visual feedback is a major parameter of performance. The experiments related to the technology do not show any significant relationship. Therefore more research in this line is required for better understanding. The screen size comparisons show that the smallest devices offer the best EERs in average, but there are devices like the iPad (that was considered as big) which offers comparable results. Then, it is not totally clear that the screen size is a definitive parameter of performance (considering that a minimum size is guaranteed). Also, the differences between modalities are not huge, although it can be said that the stylus device performance is slightly better than the finger devices performance in average. But this leads to the consideration of being both technologies belonging to the same modality

Figure 3. Genuine-only vs. Skilled-forgeries-added (for stylus-base devices)

C. Device Interoperability The capability of each device for producing interoperable templates is measured in this section. First, a comparison between devices of the same modality is done and then it is tested which is the best device for producing interoperable templates. The reference templates were obtained and tested in all the devices, nevertheless only the two devices with better interoperability results are shown in this paper. The STU was the chosen stylus device and the Note for the finger devices. As it was done for the previous experiments, the EERs are shown in the Table VIII. TABLE VIII.

B. Anti-spoofing Resistance The algorithm offers worse results using skilled forgeries signatures than only genuine signatures as it was expected. This is due to the difficulty of differentiate between a real user and a high skilled one. For instance, the fact of do not use the pressure as an input to the algorithm can be a reason for the worse results. Also, it is important to consider that the skilled forgeries were obtained with high knowledge of the genuine user’s signature, even having the genuine user as an assistant to improve the forgery quality. EER differences between genuine-only and skilled forgeries are around 10%. This is also a future research line to consider, as it maybe that by modifying the algorithm the anti-spoofing may be better achieved. Another possible conclusion for such a future work may be that advanced skilled forgeries are really a problem for this modality, and anti-spoofing mechanisms shall consider other approaches.

Interoperability results

STU template (stylus)

Note template (finger)

Note (EER: 0.98%) (Note template EER: 0.17% )

Tab (EER: 1.45%) (Tab template EER: 2.38%)

Asus (EER: 6.84%) (Asus template EER: 3.48%)

iPad (EER: 2.21%) (iPad template EER: 0.47%)

---------

Playbook (EER: 0.93%) (Playbook template EER: 1.39%)

The interoperability results show that in some cases one device offers better results using templates from other devices. Another experiment was done in order to compare the template interchange in the same device (Note), using the stylus and the finger. The results show that the templates interchange offers worse EERs than templates from the same device, but the difference between those figures are not significant enough as to deny the possibility of building interoperable systems. Also, this rejects the initial hypothesis of considering two different modalities. In fact, it defends the idea of having one single modality, but two different acquisition technologies. Figures can be seen in Table IX.

C. Device Interoperability Within genuine samples, interoperability rates are more than acceptable: in stylus-based devices is really good and in finger-based devices is affordable. But a potential problem was detected in some enrolments with fingers (for instance, better results in Tab enrolling with Note finger than with itself). This is another aspect that needs to be further studied. The results initially show that a single modality can be considered (even enrolling with stylus the EERs may get better). In summary, the results obtained show some definitive conclusions but at the same time show that more research is needed. The necessity of increasing the number of users is 178

being covered in order to achieve more accurate and reliable results. Also, there is a wide range of experiments that can be carried out including more devices, different scenarios or more than one stylus on the same device.

Technology Department in Carlos III University of Madrid (UC3M), as part of the University Group for Identification Technologies (GUTI). Furthermore, he is Member of the Spanish Standardization Subcommittee AEN / CTN71 / SC17 "Identification Cards" and Head of the Spanish Delegation at CEN / TC 224 / WG 15 "European Citizen Card" meetings.

REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13]

[14]

[15]

Dr. Oscar Miguel-Hurtado obtained his PhD in Industrial Engineering at University Carlos III of Madrid (UC3M) in 2012. He was working at the University Group for Identification Technologies, as a R&D engineer for several years. His PhD was focused in automatic identification systems by on-line handwritten signature. He is a member of the Spanish National Body for IEEE Standards subcommittee SC37.

H. Sakoe and S. Chiba, “Dynamic programming algorithm optimization for spoken word recognition”, IEEE Transactions on Acoustics, Speech, and Signal Processing, (1):43–49, 1978. Wacom STU-500 Speficications http://signature.wacom.eu/products/stu-500 Wacom Intuos 4 Specifications http://101.wacom.com/sp/intuos/ Microsoft Visual Studio Specifications http://msdn.microsoft.com/es-es/vstudio/aa718325.aspx Asus EeePC T101MT Specifications http://www.asus.es/Eee/Eee_PC/Eee_PC_T101MT/ Samsung Galaxy Note Specifications http://www.samsung.com/galaxynote/ Android SDK http://developer.android.com/sdk/ Apple iPad Specifications http://www.apple.com/ipad/specs/ Samsung Galaxy Tab Specifications http://www.samsung.com/es/microsite/galaxytab/ Blackberry Playbook Specifications http://us.blackberry.com/playbook-tablet/specifications.html Apple devices SDK http://developer.apple.com/ipad/sdk/ Blackberry Playbook specifications http://developer.blackberry.com/ Pascual-Gaspar, J. M.; Cardeñoso-Payo, V.; Vivaracho-Pascual, C.; “Practical On-Line Signature Verification”, Lecture Notes in Computer Science, 2009, Volume 5558, Advances in Biometrics, Pages 11801189. A. Mendaza-Ormaza, O. Miguel-Hurtado, R. Blanco-Gonzalo, F. J. Diez-Jimeno, “Analysis of handwritten signature performances using mobile devices”, IEEE Tans. Int. Carnahan Conf. on Security Technology, 2011. Information Technology—Biometric Data Interchange Formats—Part 7: Signature/Sign Time Series Data, ISO Standard ISO/IEC FCD 19794-7, 2006.

Aitor Mendaza-Ormaza obtained his Degree in Telecommunication Engineering (bilingual) at Carlos III University of Madrid (UC3M) in 2008. He was working at the Electronics Technology Department in Carlos III University of Madrid (UC3M) for several years, as part of the University Group for Identification Technologies (GUTI). Furthermore, he has finished an Inter-university Master's Degree in Multimedia and Communications and is working on his PhD thesis, which is focused in Biometric Algorithms applied to portable devices, mainly on Handwritten Signature Verification. Dr. Raul Sanchez-Reillo obtained his PhD in Telecommunication Engineering at Polytechnic University of Madrid (UPM) in 2000. He is currently Associate Professor at the Electronics Technology Department in University Carlos III of Madrid (UC3M). Furthermore, Dr. Sanchez-Reillo is the Director of GUTI (University Group for Identification Technology). His works in Smartcards, Information Security and Biometrics started back in 1994. Along these last years he has taken part and coordinated a great amount of projects, both in Spain and at an International Level. He is also founder of the IDTestingLab, which is a Testing Laboratory for Identification Products and Systems.

VITA

Ramón Blanco-Gonzalo obtained his Degree in Telematics Engineering at University Carlos III of Madrid (UC3M) in 2009. He is currently working in his PhD thesis, which is focused in usability in biometrics at the Electronics

179