IFSA-EUSFLAT 2009
Pattern Recognition in Blur Motion Noisy Images using Fuzzy Methods for Response Integration in Ensemble Neural Networks M. Lopez1, 2 P. Melin2 O. Castillo2 1
PhD Student of Computer Science in the Universidad Autonoma de Baja California, Tijuana, B. C., México. 2 Computer Science in the Graduate Division Tijuana, Institute of Technology Tijuana, B. C., Mexico, Email:
[email protected],
[email protected],
[email protected]
Abstract—Linear Blur Motion is one of the most common degradation functions that corrupt images. Since 1976 many researchers have tried to estimate blur motion parameters and this problem can be solved for noise free images but in the case of noisy images this can be done when the image SNR is low. In this paper, we consider pattern recognition with ensemble neural networks for the case of fingerprints; we propose the use of fuzzy methods for Response Integration in Ensemble Neural Networks for blur motion noisy images. An ensemble neural network of three modules is used; each module is a local expert on person recognition based on a biometric measure (the fingerprints). The Response Integration method of the ensemble neural networks has the goal of combining the responses of the modules to improve the recognition rate of the individual modules when the SNR rate blur motion signal increases to a high level. Keywords— Pattern Recognition, Ensemble Neural Networks, Fuzzy Logic, Ratio SNR.
1 Introduction Blur Motion occurs when there is relative motion between the camera and the object being captured [1]. When a changing scene is observed by a camera, all the classical algorithms assume that is possible to take pictures every t instant, which means that every picture is taken with a dt 0 exposure time. If that is not the case, then the exposure time (dt = T) is large enough that different points in the scene are moving far enough and consequently their corresponding projections on the image plane travel several pixels. Therefore, during the capture process of an image, at any single image point, a certain number of scene points is projected during the exposure time, each one contributing to the final brightness of the image point; this effect is shown in figure 1. More formally, during the exposure time T in front of the pixel Pi,j we could assume that they pass k scene points with brightness (C1….Ck) respectively, then the resulting brightness value for pixel Pi,j is given in equation (1), in the case it continues movement the summation is replaced by integration. This holds in general for every pixel that is moving points in the scene. It is clear that the blurring of the image exists only across the direction of the motion; this one dimensional blur is called Blur Motion.
ܲǡ ൌ
ͳ ܥ ሺͳሻ ܭ ୀଵ
ISBN: 978-989-95079-6-8
The Result of blur motion is shown in figure 1b where an image consistent of different value pixels is shown in figure 1a and then the blurred image is shown in figure 1b.
Figure 1: a) Original Image without noise, b) Blur Motion with 20 pixels distance d and 50 degrees angle . The blur motion can be described mathematically as the result of a linear filter b(x, y) = i(x, y)*h(x, y) where i is the theoreticalcal image taken with an exposure time Te=0, b is the real blurred image and h the point spread function (PSF). Given an angle = and the length d = Vo x Te, which is the number of scene points that affect a specific pixel, the point spread function of blur motion is given in equation (2). ͳ ݄ሺݔǡ ݕሻ ൌ ቊ ǡ Ͳ ȁݔȁ ݀
כሺߙሻ ݕൌ ሺߙሻ ݀ כሺʹሻ ݀ Ͳǡ In practical terms, this mean computing accurate estimates for the two parameters of the blur motion PSF, namely the length, d, and the angle, . From these quantities, the relative velocity at this point can be easily recovered knowing the exposure time. In this paper for the experimental results we use fingerprint images and adding different levels of blur motion noise from 10 to 90 pixels in distance d and the angle is zero. The measurement for noise in an image is the signal-to-noise ratio, SNR. The SNR measures the relative strength of the signal in a blurred and noisy image to the strength of the signal in a blurred image without noise. A SNR of 16 db is a low noise level, while a SNR of 4 db is a high noise level [1].
2 Noise and Registration Error 2.1 Signal-to-noise ratio Noise is present in all digital imaging systems due to a number of sources such as photon shot noise, readout noise, dark current noise and quantization noise. While some sources can be effectively suppressed, such as the dark
809
IFSA-EUSFLAT 2009 current noise by cooling and the quantization noise by using more bits, others not. Photon shot noise, for example, is unavoidable due to the particle nature of light. Readout noise increases with a higher readout rate, which is desirable in high-speed cameras. In equation 3 the calculus of the Variance and Standard deviation of the image are shown [2]. ଵ
ߪூ ଶ ൌ ݎܽݒሺܫଵ െ ܫଶ ሻǡ ܫ݁ݎ݄݁ݓଵ ൌ ܫ ߟଵ ǡ ܫଶ ൌ ܫ ߟଶ ሺ͵ሻ ଶ
ߪூ = the Standard deviation of the Fingerprint Image Original. = ݎܽݒthe variance between the Fingerprint Image Original and The Fingerprint Image with Blur Motion Noise. ߟଵ ൌ Ͳ, is the noise to the Fingerprint Image Original. ߟଶ = blur motion noise 10 to 90 distance pixels. ܫଶ , is the Fingerprint Image with Blur Motion Noise.
Table I: The Average SNR in db of the Fingerprint Images. Average SNR in db Person N° Fingerprint Image P1 P2 P3 P4 P5 P6 P7 P8 P9 P10
Blur Motion Distance Pixels
10 10.9 8.36 8.01 7.67 11.6 7.29 8.21 10.6 12.7 8.94
20 7.57 8.02 7.02 5.75 10.62 7.28 6.11 10.07 12 6.43
30 8.08 7.52 6,76 5.57 10.07 6.88 6.26 9.4 11.65 6
40 7.65 7.38 6.56 5.46 9.77 6.75 6.18 9.08 11.35 5.79
50 7.43 7.16 6.44 5.26 9.47 6.58 5.93 8.8 11.7 5.52
60 7.28 7.01 6.34 5.17 9.28 6.46 5.85 8.56 10.9 5.33
70 7.11 6.89 6.21 5.07 9.08 6.33 5.79 8.47 10.81 5.24
80 6.99 6.78 6.10 5.01 8.88 6.20 5.67 8.15 10.6 5.14
90 6.89 6.68 5.98 4.93 8.71 6.09 5.57 7.97 10.4 5.03
In figure 3 we show the plot of the Average SNR of the Person Fingerprint with the blur motion level noise from 10 to 90 distance pixels.
The Signal-to-Noise Ratio (SNR) of an image I is then computed as: ܴܵܰ ൌ ͳͲሺ
௩ூ ఙమூ
ሻሺͶሻ
The combined effect of these noise sources is often modeled from two images of the same scene captured under the same degradation is shown in figure 2. Figure 3: Average SNR with blur motion noise.
3 Ensemble Neural Networks Architecture Figure 2: Image degradation by blur and additive noise. 2.2 Calculus of the SNR for Fingerprint Images In this paper we use fingerprint images from the FCV2000 database of the Biometric System Laboratory at the University of Bologna [3, 4]. The image size is 300 pixels wide and 300 pixels high with a resolution of 500 ppi, and of a gray scale representation. The fingerprints were acquired by using a low-cost optical sensor; up to four fingers were collected from each volunteer (forefinger and middle finger of both the hands). The database is 10 fingers wide (w) and 8 impressions per finger deep (d) (80 fingerprints in all); the acquired fingerprints were manually analyzed to assure that the maximum rotation is approximately in the range [-15°, 15°] and that each pair of impressions of the same finger have a non-null overlapping area. We added Blur Motion Level Noise from 10 to 90 distance pixels to all 80 fingerprint images, based on equations (3) and (4) the calculus of the Average SNR in decibels for the Fingerprint Image of each of the person obtained is shown in table 1.
3.1 ANN´s Architecture The Ensemble ANN´s architecture consists of three main modules [5, 6], in which each of them in turn consists of a set of neural networks trained with the same data (fingerprints image), If we now using different parameters for each module of the Ensemble Neural Networks, for module 1 the used parameters are learning rate =.001 , Goal Error=.001, for the module 2 learning rate =.0001 , Goal Error=.0001, for the module 3 the learning rate =.0001, and Goal Error=.00001, using 2 hidden layers, 36 neurons in the first layer and 17 neurons in the second layer for each module of the Ensemble Neural Network and is shown in figure 4.
Figure 4: Ensemble Neural Networks with blur Motion Level Noise Added.
ISBN: 978-989-95079-6-8
810
IFSA-EUSFLAT 2009 The procedure to perform the tests was first to train the modules of the ensemble neural networks with the databases of the fingerprints without noise, in total using 8 fingerprints of 10 people, that is 80 fingerprints in total, until being able to find which architecture of the ensemble neural network responded better to arrive to the desired error. We Added Blur Motion Level Noise from 10 to 90 distance pixels to the Fingerprint Input, to obtain a blur motion image that went into each of three modules ensemble neural networks. The output of each of the ensemble neural networks was obtained response integration with type-1 or type-2 fuzzy logic, and finally with the decision module obtained the output fingerprint. 3.2 Response Integration with Type-1 Fuzzy Logic We also used a Type-1 Fuzzy Inference System as method of response integration of the ensemble neural network output, considering as input three linguistic variables, Activation Low, Activation Medium, and Activation High, and one output linguistic variable, Winning Activation of the three modules. We show below one of the rules for the fuzzy inference system. If (Module1 is ActMod1Low) and (Module2 is ActMod2Medium) and (Module3 is ActMod3Medium) then (Winner Module is Module3) Three membership functions were used for each linguistic variable (input and output) of the triangular type, to be managed in a range from 0 to 1. We show in figure 5 the membership functions designed using the editor of the fuzzy logic toolbox of MATLAB [14].
managed in a range from 0 to 1. We show in figure 6 the membership functions designed using the editor of IT2FUZZY fuzzy logic toolbox [12, 13].
Figure 6: The Input Membership function and The Output Membership function of Type-2 Fuzzy Logic.
4 Simulation Results 4.1 Response Integration with Type-1 Fuzzy Logic Once the Ensemble Neural Network is trained, the fuzzy inference system integrates the outputs of the modules. We used the same 80 persons images to which we had applied different levels of noise with blur motion, the type-1 inference systems give an output for the stage of the final decision, and we show the result if the fingerprint input was recognized. We show in figure 7 the simulation results.
Figure 7: Simulation results for the fingerprints using the type-1 Fuzzy Inference System (blur motion 50 distance pixels).
Figure 5: The Input Membership function and The Output Membership function of Fuzzy Logic Type-1. Once the Ensemble Neural Network is trained, the fuzzy inference system integrates the outputs of the modules. We used the same 80 people’s images to which we had applied noise with blur motion, and the type-1 fuzzy inference system gives an answer for the stage of the final decision. 3.3 Response Integration with Type-2 Fuzzy Logic We used a Type-2 Fuzzy Inference System as method of response integration of the ensemble neural network output, considering as input three linguistic variables, Activation Low, Activation Medium, and Activation High, and one output linguistic variable, Winning Activation of the three modules, used the same rules of the type-1 fuzzy logic. Three membership functions were used for each linguistic variable of the input and output of the Gaussian type ISBN: 978-989-95079-6-8
4.2 Response Integration with Type-2 Fuzzy Logic In the same way when the Ensemble Neural Network is trained, the fuzzy inference system integrates the outputs of the modules. We used the same 80 persons images to which we had applied different levels of noise with blur motion, the type-2 inference systems give an output for the stage of the final decision, and show the result if the fingerprint input was recognized. We show in figure 8 the simulation results.
Figure 8: Simulation results for the fingerprints using the type-2 Fuzzy Inference System (blur motion 50 distance pixels).
811
IFSA-EUSFLAT 2009 4.3 Comparison Pattern Recognition Blur Motion Noisy Images using Fuzzy Methods for Response Integration in Ensemble Neural Networks The Response Integration method of the ensemble neural networks has the goal of combining the responses of the modules to improve the recognition rate of the individual modules when the blur motion signal increases to high level 10 to 90 distance pixels. In Table 2 we show a Comparison between Response Integration with Type-2 and Type-1 Fuzzy logic when the blur motion level noise increases and the Average SNR in db decrease.
noise increases and the down side the Average SNR in db decrease for each of the Person Fingerprint Images.
Table II: Comparison Response Integration Type-2 and Type-1 Fuzzy Logic with Blur Motion Level Noise and the Average SNR in db of the Fingerprint Images. # Person Fingerprint Average SNR P1 db Fuzzy Type-2 % Fuzzy Type-1 % Average SNR P2 db Fuzzy Type-2 % Fuzzy Type-1 % Average SNR P3 db Fuzzy Type-2 % Fuzzy Type-1 % Average SNR P4 db Fuzzy Type-2 % Fuzzy Type-1 % Average SNR P5 db Fuzzy Type-2 % Fuzzy Type-1 % Average SNR P6 db Fuzzy Type-2 % Fuzzy Type-1 % Average SNR P7 db Fuzzy Type-2 % Fuzzy Type-1 % Average SNR P8 db Fuzzy Type-2 % Fuzzy Type-1 % Average SNR P9 db Fuzzy Type-2 % Fuzzy Type-1 % Average SNR P10 db Fuzzy Type-2 % Fuzzy Type-1 %
Blur Motion Level Noise 40 50 60 7.65 7.43 7.28
10 10.97
20 7.75
30 8.08
70 7.11
80 6.99
90 6.89
100
87.5
87.5
87.5
87.5
62,5
62.5
62.5
62.5
87.5
87.5
87.5
87.5
75
62.5
62.5
62.5
62.5
8.33
8.02
7.52
7.38
7.16
7.01
6.89
6.78
6.68
100
100
100
100
87.5
100
87.5
75
62.5
50
37.5
87.5
100
87.5
75
75
50
50
8.01
7.02
6.76
6.56
6.44
6.34
6.21
6.10
5,98
100
100
87.5
87.5
62.5
62.5
37.5
25
12.5
100
100
87.5
75
75
50
0
0
0
7.67
5.75
5.57
5.46
5.26
5.17
5.075
5.01
4.94
100
100
87.5
87.5
75
75
62.5
62.5
25
87.5
87.5
87.5
87.5
75
75
62.5
62.5
37.5
11.65
10.62
10.08
9.77
9.47
9.28
9.08
8.89
8.71
100
100
100
100
100
87.5
87.5
75
50
100
100
100
100
100
87.5
75
75
50
7.29
7.28
6.88
6.75
6.58
6.46
6.33
6.21
6.1
100
100
100
100
100
100
100
87.5
75
100
100
100
100
100
100
100
87.5
75
8.21
6.11
6.26
6.18
5.93
5.85
5.79
5.67
5.57
100
100
100
87.5
87.5
87.5
62.5
50
37.5
100
100
100
75
75
87.5
62.5
50
37.5
10.64
10.07
9.40
9.09
8.8
8.56
8.47
8.15
7.97
100
100
100
87.5
87.5
87.5
62.5
50
37.5
100
100
100
75
75
87.5
62.5
50
37.5
12.76
12
11.65
11.35
11.7
10.9
10.81
10.6
10.42
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
8.21
6.11
6.26
6.18
5.93
5.85
5.79
5.67
5.57
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
In Figure 9 we show in the up side the response integration of the type-2 and type-1 fuzzy system when the blur motion
ISBN: 978-989-95079-6-8
812
IFSA-EUSFLAT 2009
ISBN: 978-989-95079-6-8
813
IFSA-EUSFLAT 2009
Figure 9: Response Integration Type-2 and Type-1 Fuzzy logic with the blur Motion Level Noise 10 to 90 Distance Pixels and Average SNR in db of each 10 Person Fingerprint Image.
5 Conclusions Based on the experimental results, we can conclude that the behavior of the type-2 and type-1 fuzzy systems, as methods of response integration of ensemble neural networks for the fingerprints when added blur motion level noise increase and the Average SNR of the Fingerprint decrease, we obtain excellent results. The identification rates obtained are between 37.5% to 100%for response integration type-2 fuzzy logic and 0% to 100% for response integration with type-1 fuzzy logic. The difference between the type-1 fuzzy and type-2 fuzzy inference systems for response integration of ensemble neural networks is appreciated when we uses the blur motion level noise. We think that there is an advantage in using a type-2 fuzzy inference system to manage the uncertainty of the knowledge base in pattern recognition problems. Future work will include, testing with more kinds of noise, other methods of response integration, using feature extraction, and image compression, with the goal of improving the identification rate.
Acknowledgment We would like to express our gratitude to CONACYT, Universidad Autonoma de Baja California and Tijuana Institute of Technology for the facilities and resources granted for the development of this research project. References [1]
Felix Krahmer_, Youzuo Lin, Bonnie McAdoo, Katharine Ott, Jiakou Wang, David Widemannk, Brendt Wohlberg, Blind Image Deconvolution: Motion Blur Estimation, August 2006.
[2]
Tuan Q. Pham, Lucas J. van Vliet, Klamer Schutte, Influence of signal-to-noise ratio and point spread function on limits of super resolution, Image Processing, Algorithms and Systems IV, Vol. 5672, pp. 169-180, 2005.
ISBN: 978-989-95079-6-8
[3]
D. Maltoni, D. Maio, A.K. Jain, S. Prabhakar, The full FVC2000 and FVC2002 databases are available in the DVD included in: Handbook of Fingerprint Recognition, Springer, New York, 2003.
[4]
D. Maio, D. Maltoni, R. Cappelli, J. L. Wayman and A. K. Jain, "FVC2004: Third Fingerprint Verification Competition", Proc. International Conference on Biometric Authentication (ICBA), pp. 17, Hong Kong, July 2004.
[5]
M. Lopez, P. Melin: Response integration in Ensemble Neural Networks using interval type-2 Fuzzy logic. IJCNN 2008: 1503-1508
[6]
O. Mendoza, P. Melin, G. Licea, Modular Neural Networks and Type2 Fuzzy Logic for Face Recognition, In Marek Reformat, editor, Proceedings of NAFIPS 2007, Number 1, pages CD Rom, San Diego, June 2007.
[7]
J. Urias, D. Solano, .M. Soto, M. Lopez, P. Melin, ¨Type-2 Fuzzy Logic as a Method of Response Integration in Modular Neural Networks¨. IC-AI 2006: 584-590.
[8]
P. Melin, F. González, G. Martínez: Pattern Recognition Using Modular Neural Networks and Genetic Algorithms. IC-AI 2004: 7783.
[9]
P. Melin, J. Urias, D. Solano, M. Soto, M. Lopez, O. Castillo: Voice Recognition with Neural Networks, Type-2 Fuzzy Logic and Genetic Algorithms. Engineering Letters 13(2): 108-116 (2006).
[10] P. Melin, A. Mancilla, M. Lopez, D. Solano, M. Soto, O. Castillo, ¨Pattern Recognition for Industrial Security using the Fuzzy Sugeno Integral and Modular Neural Networks¨, WSC11 11th Online World Conference on Soft Computing in Industrial Applications September 18-October 6, 2006. [11] P. Melin and O. Castillo, 2005, ¨Hybrid Intelligent Systems for Pattern Recognition using Soft Computing¨, ISBN 3-540-24121-3 Springer Berlin Heidelberg New York. [12] J. R. Castro, O. Castillo, P. Melin, L.G. Martinez, S. Escobar, I. Camacho, Building Fuzzy Inference Systems with Interval Type-2 Fuzzy Logic Toolbox Number 1 in Studies in Fuzziness and Soft Computing, 6, pages 53-62, Springer-Verlag, Germany, 1 edition, June 2007. [13] J. R. Castro, O. Castillo, P. Melin, An Interval Type-2 Fuzzy Logic Toolbox for Control Applications; In Proc. FUZZ-IEEE 2007. [14] MATLAB Trade Marks, ©1994-2007 by The Math Works, Inc. [15] S. Farsiu, D. Robinson, M. Elad and P. Milanfar. Fast and robust multi-frame super-resolution. IEEE Trans. on Image Processing, 13(10):1327-1344, 2004. [16] R.C. Hardie, K.J. Barnard and E.E. Armstrong. Joint map registration and high-resolution image estimation using a sequence of undersampled images. IEEE Trans.on Image Processing, 6(12):1621– 1633, 1997. [17] R. Neelamani, H. Choi and R. Baraniuk. Forward: Fourier-wavelet regularized deconvolution for Ill-Conditioned Systems. In IEEE Trans. Signal Processing, 52(2):418–433, 2004. [18] J. Biemond. Iterative methods for image deblurring. Proceedings of the IEEE, 78(5):856–883, 1990. [19] M.Ebrahimi Moghadam, M.Jamzad, Linear motion blur parameter estimation in noisy images using fuzzy sets and power spectrum, EUROSIP Journal on applied signal processing, Article number 68985, Volume 2007, pages 862–866.. [20] Y. Yitzhaky, I. Mor, A. Lantzman, and N. S. Kopeika. Direct method for restoration of motion blurred images. Journal of the Optical Society of America. A, Optics and image science, 15(6):1512–1519, 1998.
814
