Palmprint and Palm Vein Multimodal Fusion Biometrics ... - Springer Link

0 downloads 0 Views 907KB Size Report
In this paper, the fusion identification method of the palmprint and palm vein based ... p5 p6. (a) LBP operator. (b) NBP pixel sequence. Fig. 1. LBP operator and NBP pixel ... 3, we can see that after operating by NBP, the network structure.
Palmprint and Palm Vein Multimodal Fusion Biometrics Based on MMNBP Sen Lin1(&), Ying Wang1, Tianyang Xu1, and Yonghua Tang2 1

School of Electronic and Information Engineering, Liaoning Technical University, Huludao 125105, Liaoning, China [email protected] 2 Computer Vision Group, Shenyang University of Technology, Shenyang 110870, Liaoning, China

Abstract. This paper presents a multi-biometrics recognition method based on the fusion of palmprint and palm vein. Firstly, the traditional LBP method is improved, a novel algorithm called neighbor based binary pattern (NBP) is presented, which uses the relationship of gray value between adjacent pixels in the local area to encode the image. Secondly, the images of palm vein and palmprint are subdivided into several uniform size blocks, the gray mean value of each block is calculated. Furtherly, the multi-block mean image is encoded by the NBP method, which is called multi-block mean neighbor based binary pattern (MMNBP), and the feature fusion operation is implemented. Finally, the Hamming distance is used for matching. The comparison experiments are carried out with the current typical and popular approaches in the PolyU contact public database and self-built non-contact database. The experimental results indicate the superiority and effectiveness of the approach, which has good application prospect. Keywords: Texture feature  Neighbor based binary pattern (NBP)  Palmprint and palm vein  Feature fusion

1 Introduction With the rapid development of information technology, the requirements for identity recognition become higher, so biometric identification technology is becoming more and more popular for its safety and convenience [1]. However, there are some shortcomings in the typical biometrics methods, for example, the fingerprint is easy to be forged; the methods based on face recognition are easy to be interfered by gesture and decoration. In the last few years, the biometric identification technology based on the palmprint and palm vein have shown significant advantages [2]. Moreover, the identification of palmprint and palm vein can be designed in only one non-contact system, which is more friendly and widely accepted. When reviewing the literature of current research, the recognition methods of palmprint and palm vein mainly include the following categories: the first is the method based on structural features [3], the second is based on subspace [4], and the third is based on invariant features [5]. The method based on texture is a very active research © Springer International Publishing AG 2016 Z. You et al. (Eds.): CCBR 2016, LNCS 9967, pp. 326–336, 2016. DOI: 10.1007/978-3-319-46654-5_36

Palmprint and Palm Vein Multimodal Fusion Biometrics Based on MMNBP

327

direction, which is used to extract the global or local statistical information of the palmprint or palm vein image as features. For example, a new feature input space and a descriptor that operates in the local line-geometry space is proposed by Luo [6], and a new image descriptor called LLDP (local line directional patterns) is presented for palmprint recognition. The effects of two methods as LBP (local binary patterns) and LDP (local derivative patterns) were studied in detail for palm vein recognition [7]. The texture analysis methods have stronger versatility, and they are fit the own characteristics of the palmprint or palm vein images intuitively, so they have a broad prospect in application [8]. Obviously, identification by the fusion of palmprint and palm vein feature is very simple and practical [9], because it can be designed in the same acquisition device. In this paper, the fusion identification method of the palmprint and palm vein based on the neighbor binary pattern (NBP) is proposed. To improve the traditional LBP coding method, the NBP encoding is informed according to the relations of gray values based on the local neighbor pixels of the images. The pictures of palmprint and palm vein are divided into several uniform region blocks, and the gray mean of each block is calculated namely multi-mean NBP (MMNBP) to enhance the robustness of the algorithm. After that, we connect the different features by using fusion method, and finally we do the matching by using Hamming distance.

2 Principle Method 2.1

LBP Algorithm and NBP Algorithm

The traditional LBP algorithm is mainly concerned with the relationship between the local neighbor pixels and the center pixel [10]. The formulas are described as follows: LBPP;R ðxc ; yc Þ ¼

p1 X

2i  sðpi  pc Þ

ð1Þ

i¼0

( sðpi  pc Þ ¼

1; pi  pc  0 0; pi  pc \0

ð2Þ

Where (xc; yc ) is the coordinate of the center pixel and the gray value is pc . pi is the gray value of the neighborhood; i is the neighborhood position; R is the neighborhood radius; P is the number of pixels in the neighborhood. Figure 1(a) shows the LBP encoding process when P = 8, R = 1. NBP is an improved texture description method, which mainly focus on the relationships of gray values among the neighbor pixels [11]. The method of the pixel arrangement in the 3  3 window is shown in Fig. 1(b). There are three steps for NBP encoding: Step 1, in 3  3 window, based on the center pixel, 8 points around the center pixel are extracted and arranged in a line by taking the pixel in the left corner as the starting point;

328

S. Lin et al.

78

99

50

54

54

49

57

12

13

1

Threshold

1

1 1

Binary coding: 11000011 LBP value 195

0 0

0

p7

p6

p0

p5 p4

0

p1

p2

p3

(b) NBP pixel sequence

(a) LBP operator

Fig. 1. LBP operator and NBP pixel sequence

Step 2, starting from the pixel on the most left side, each pixel can be operated as follows: the gray values are compared between the current pixel and the next neighbor pixel on the right by the formula as: ( si þ 1 ðpi þ 1  pi Þ ¼

1; pi þ 1  pi [ 0 0; pi þ 1  pi  0

i ¼ 0; 1; 2; . . .; 6

ð3Þ

Specially, ( s0 ðp0  p7 Þ ¼

1; p0  p7 [ 0 0; p0  p7  0

ð4Þ

So a binary code of 8 bit can be formed. Step 3, the binary encoding bit string is transformed into a decimal number, which is the NBP value of center pixel, the formula is: DNBP ¼

7 X

si  2i

ð5Þ

i¼0

The schematic of NBP encoding principle and the examples are showed in Fig. 2. Analysis window 4

6

7

3

5

10

5

4

9

Extraction of neighbors

4

6

7

10 9

4

5

3

1

0

1

0

NBP Operator 26

NBP image

Conversion from binary to decimal

0

0

0

NBP code:00011010 NBP number :26

Fig. 2. NBP coding principle

1

Palmprint and Palm Vein Multimodal Fusion Biometrics Based on MMNBP

2.2

329

NBP Feature Map

When an image of palmprint or palm vein is encoded by NBP, the decimal number can be regarded as the gray value, which can form NBP feature map, as shown in Fig. 3. Figure 3(a) is the ROI (region of interest) of palmprint and the corresponding NBP feature map, Fig. 3(b) is the ROI of palm vein and the corresponding NBP feature map. As shown in Fig. 3, we can see that after operating by NBP, the network structure of palmprint and palm vein will be clearer, the texture features are obviously prominent, which is beneficial for subsequent identification.

(a) Palmprint image and its NBP feature map (b) Palm vein image and its NBP feature map Fig. 3. Palmprint and palm vein images and NBP feature maps

2.3

MMNBP and Feature Extraction of Palmprint and Palm Vein

The texture information is rich in palmprint and palm vein images, which is proper for the use of NBP feature extraction, but the dimension is high when NBP operation is implemented directly on the whole image. Meanwhile the comparisons of original NBP single pixel are more easily affected by noise. Therefore, this paper proposes a feature extraction method of palmprint and palm vein based on the multi-block mean neighbor binary pattern (MMNBP). First of all, the image of palmprint and palm vein is processed by ROI extraction and gray enhancement, then a series of uniform block are formed, which is to transform an image matrix of v whose size is M  M into image blocks, as follow: 2

V11 6 V21 V ¼6 4 Vk1

V12 V22  Vk2

   

3 V1k V2k 7 7  5 Vkk

ð6Þ

Here each block Vij (i, j = 1,2,…,k) is a m  m matrix (M = k  m). Secondly, the gray value of each block can be calculated by the following formula: uij ¼

m X m 1 X fij ðx; yÞ ði; j ¼ 1; 2;    ; kÞ m2 x¼1 y¼1

ð7Þ

Here fij (x, y) shows the gray value of point (x, y) in the block. The image block average matrix of k  k size is formed by using the gray mean values of all blocks:

330

S. Lin et al.

2

u11 6 u21 I¼6 4 uk1

u12 u22  uk2

   

3 u1k u2k 7 7  5 ukk

ð8Þ

The multi-block mean calculation is shown in Fig. 4 (take palm vein image as an example). The advantages of the block mean calculation can be summarized as follows: (1) The block mean calculation can further reduce the noise effect of the gray value comparison between single pixels. (2) Reduce the possible impact of the image rotation caused by the acquisition, and enhance the robustness of the method. (3) It can reduce the original data dimension and improve the operation efficiency, which is simple and easy to operate.

Multi-block mean value calculation

26

38

62

29

40

27

50

33

35

30

22

41

62

71

80

70

Fig. 4. Multi-block mean value calculation

Finally, NBP operation is implemented by using the matrix of the multi-block mean image, and the final MMNBP encoding result of the palmprint or the palm vein is obtained.

3 Palmprint and Palm Vein Feature Fusion and Matching 3.1

Feature Fusion

Image fusion is an effective method in multi-biometric recognition [12]. There are three main types of image fusion at present, and among them the application of feature level fusion is the most common [13]. The pictures of palmprint and palm vein all have rich texture information, from this perspective, they have a certain similarity, so they are suitable for the fusion operation. At the same time, the feature fusion of palmprint and palm vein can provide more abundant information for identity identification, and significantly increase the difficulty of forgery, which can effectively improve the security and reliability of the system. If the MMNBP binary encoding of a sample palmprint image is SPP-MMNBP, the MMNBP binary encoding of a sample palm vein image is SPV-MMNBP, then the fusion feature SF-MMNBP is the connection of two codes, which can be defined as:

Palmprint and Palm Vein Multimodal Fusion Biometrics Based on MMNBP

331

SF-MMNBP ¼ ½SPP-MMNBP SPV-MMNBP 

ð9Þ

Since the encoding of the binary is the type of bit in both SPP-MMNBP and SPV-MMNBP, so there is no need to implement feature normalization operation, which is more convenient.

3.2

Feature Matching

During the match, the Hamming distance between MMNBP encoding after feature fusion are calculated. If there are two MMNBP codes called SF-MMNBP1 and SF-MMNBP2 respectively, the forms of bit string are: sF-MMNBP1 ¼ x1 x2    xN

ð10Þ

sF-MMNBP2 ¼ y1 y2    yN

ð11Þ

Here x, y are 0 or 1. The Hamming distance between them is defined as: N P

RHD ðSF-MMNBP1 ; SF-MMNBP2 Þ ¼ i¼1

xi  yi N

ð12Þ

Symbol  indicates the XOR operation, and N is the length of the bit string. During the match, the similarity degree of two features is measured by using Hamming distance RHD. The RHD value ranges from 0 to 1, and a smaller value indicates a higher similarity. A threshold t is set in the specific identification. When RHD satisfies follow: RHD \t

ð13Þ

The two images are from the same person, and will be accepted, otherwise be refused.

4 Experimental Results The whole identification system of dual modality fusion of palmprint and palm vein can be described as Fig. 5:

4.1

Evaluation Index Definition and Algorithm Test Environment

In this paper, the experimental evaluation algorithm of intra-class and inter-class matching has been used [8]. We can select the appropriate threshold value, and

332

S. Lin et al.

Input image

preprocessing

Result

Multi-block Mean value calculation

NBP coding

Features fusion

database

Hamming Distance matching

Decision

Fig. 5. Flow chart of the recognition system

complete the identification according to the matching principle above by drawing the distribution curve of intra-class and inter-class matching. EER (error rate equal) is adopted as the evaluation index. Generally speaking, the value of EER is smaller, the effect of recognition is better [9]. The EER can be obtained as follow: take the FAR (false accept rate) as the horizontal coordinate and FRR (false rejection rate) as the vertical coordinate in the rectangular coordinate system of the same plane to draw ROC curve (receiver operating characteristics curve), the point at which the FRR equal to FAR is EER. The calculation formulas of FAR and FRR are as follow: WFAR ¼

VA  100 % VJ

ð14Þ

WFRR ¼

VE  100 % VH

ð15Þ

Among them, VJ is the login times of fake users; VH is the registration number of legitimate users; VA is the number of false acceptance, that is, the number of accepting the fake user into the system; VE is the number of false rejection, which is the number of rejecting the legitimate users out of the system. The test environment of the algorithm in this paper is the desktop computer which has been installed MATLAB software (version number: 2011b), and the CPU frequency is 3.0 GHz with 12 GB RAM.

4.2

Experimental Database and Results

To fully test the effectiveness of the algorithm, we use two image database described as follows: (1) PolyU database. In this paper, the NIR (near infrared) of 850 nm band was adopted to collect palm vein images for 100 people, 5 ROI images per person were used as experimental samples. The white LED (light emitting diode) was selected to obtain palmprint images, the sample number was also 100 people, and 5 images per person [14]. (2) Self-built database. Dual optical CCD images sensor camera AD080GE was used as acquisition equipment. A non-contact database of palm vein was built. The NIR LED which wavelength is 850 nm has been selected as the light source indoor, and the images of 50 persons were obtained with 10 images of right hand palm

Palmprint and Palm Vein Multimodal Fusion Biometrics Based on MMNBP

333

vein per person. Then we chose white LED as the light source for palmprint collection, the number of samples was also 50 people with 10 images per person. The sample images of two databases are given in Fig. 6 and ROI size is 128  128. Different block division is selected, and the EER of every case for the fusion is calculated, as shown in Table 1. We can see that if the block is too large or too small, the EER will rise. This phenomenon can be interpreted as follow: the size of the block must be adapted to palmprint or palm vein texture, too large or too small blocks will affect the EER. And we can also find that the EER of self-built database is higher than the PolyU, this because the environmental reasons of image collection, or there may be some errors in the ROI extraction. Next step we will improve on these.

(a) Palm vein (PolyU database)

(b) Palmprint (PolyU database)

(c) Palm vein (Self-built database)

(d) Palmprint (Self-built database)

Fig. 6. Examples of two databases Table 1. EER of the different blocks for fusion method (%) Block size 22 44 88 16  16 32  32 64  64

PolyU database Self-built database 0.1001 3.3838 0.0008 1.9745 0 1.4103 0.0097 1.7202 2.5288 5.1093 27.2662 22.4902

The matching curve of intra-class and inter-class in PolyU database is given in Fig. 7 (a), when the blocks are subdivided by 8  8 pixels, the times of intra-class and inter-class is 1000 and 123750 respectively. The threshold is set at the point of intersection of two curves, which is t = 0.5598. The result in self-built database is given in Fig. 7(b), the times of intra-class and inter-class is 2250 and 122500 respectively, and t = 0.6132. The curve of ROC is given on the corresponding database in Fig. 8. To reflect the advantages of the proposed method, the MMNBP (before and after fusion) is compared with LBP and some typical and popular approaches currently [4, 8] on the palmprint and palm vein database, as shown in Table 2. We can see that our fusion method is the best.

334

S. Lin et al.

(a) PolyU database

(b) Self-built database

Fig. 7. Matching curve

Fig. 8. ROC curve

Table 2. EER comparison of different methods (%) Algorithm PolyU database Palmprint Palm vein 2DGabor 0.3023 0.8068 2DPCA 0.4981 2.3006 WGS 0.7205 2.3889 SURF 3.3161 3.1397 LBP 1.8938 0.7106 MMNBP 0.4364 0.6983 MMNBP 0 (fusion)

Self-built database Palmprint Palm vein 24.0487 3.6111 10.6574 2.5912 11.5655 3.4744 7.4488 3.3312 13.2341 1.9873 13.0294 1.6036 1.4103 (fusion)

The comparison of MMNBP execution time before and after the fusion is given in Table 3. We can see that the execution time of the method is satisfactory.

Palmprint and Palm Vein Multimodal Fusion Biometrics Based on MMNBP

335

Table 3. Execution time of the algorithm PolyU database Palmprint Palm vein Features extraction time (s) 0.094 0.095 Matching time (s) 0.015 0.015 Total time (s) 0.109 0.11

Fusion 0.165 0.016 0.181

Self-built database Palmprint Palm vein 0.092 0.097 0.016 0.017 0.108 0.114

Fusion 0.164 0.018 0.182

5 Conclusions and Discussion From the perspective of texture image analysis, the operation of texture neighbor pattern is adopted in this paper, and the identification method of dual modality and multi-biometric feature fusion of palmprint and palm vein based on MMNBP is proposed. The core of this approach is NBP operation, by improving the encoding method of traditional LBP and combining the method of multi-block mean operation, the NBP is further enhanced. The experimental results in the two databases show that the MMNBP method of dual modality fusion could obtain the minimum EER of 0 and 1.4103 %, and the recognition time is only 0.181 s and 0.182 s. These show that the method is straightforward and feasible, which has a practical prospect. Acknowledgement. This work is supported by (1) General Scientific Research Project of Liaoning Provincial Committee of Education (L2014132); (2) Natural Science Foundation of Liaoning Province of China (2015020100).

References 1. Unar, J.A., Seng, W.C., Abbasi, A.: A review of biometric technology along with trends and prospect. Pattern Recogn. 47(8), 2673–2688 (2014) 2. Wu, W., Yuan, W.Q.: A survey of palm-vein image recognition. J. Image Graph. 18(10), 1215–1224 (2013) 3. Jia, W., Hu, R.X., Lei, Y.K., et al.: Histogram of oriented lines for palmprint recognition. IEEE Trans. Syst. Man Cybern. Syst. 44(3), 385–395 (2014) 4. Wu, W., Yuan, W.Q., Lin, S., et al.: Selection of topical wavelength for palm vein recognition. Acta Opt. Sin. 32(12), 1211002 (2013) 5. Kang, W.X., Liu, Y., Wu, X.Q.: Contact-free palm-vein recognition based on local invariant features. PLoS ONE 9(5), e97548 (2014) 6. Luo, Y.T., Zhao, L.Y., Zhang, B., et al.: Local line directional pattern for palmprint recognition. Pattern Recogn. 50, 26–44 (2016) 7. Mirmohamadsadeghi, L., Drygajlo, A.: Palm vein recognition with local texture patterns. IET Biometrics 3(4), 198–206 (2014) 8. Wu, W., Yuan, W.Q., Lin, S., et al.: Fast palm vein identification algorithm Based on grayscale surface matching. Acta Opt. Sin. 33(10), 1015004 (2013) 9. Zhang, D., Guo, Z.H., Lu, G.M.: Online joint palmprint and palmvein verification. Expert Syst. Appl. 38(3), 2621–2631 (2011)

336

S. Lin et al.

10. Zhao, Y.: Theories and applications of LBP: a survey. In: 7th International Conference on Advanced Intelligent Computing Theories and Applications, pp. 112–120. IEEE Press, Zhengzhou (2011) 11. Hamouchene, I., Aouat, S.: A cognitive approach for texture analysis using neighbors-based binary patterns. In: 13th International Conference on Cognitive Informatics & Cognitive Computing, pp. 94–99. IEEE Press, London (2014) 12. Ahmad, M.I., Woo, W.L., Dlay, S.: Non-stationary feature fusion of face and palmprint multimodal biometrics. Neurocomputing 177, 49–61 (2016) 13. Zhang, Y.Q., Sun, D.M., Qiu, Z.D.: Hand-based feature level fusion for single sample biometrics recognition. In: 1st International Workshop on Emerging Techniques and Challenges for Hand-Based Biometrics, pp. 1–4. IEEE Press, Istanbul (2010) 14. Guo, Z.H., Zhang, D., Zhang, L., et al.: Feature band selection for online multispectral palmprint recognition. IEEE Trans. Inf. Forensics Secur. 7(3), 1136–1139 (2010)