A Neural Network for Recognizing Characters Extracted from Moving Vehicles Jang-Hee Yoo and Byung-Tae Chun
Arti cial Intelligence Division Systems Engineering Research Institute P.O. Box 1, Yoosung, Taejeon, 305-600, Korea email:
[email protected] Dong-Pil Shin
Korea Information Networks Yoido-Dong 14-24, Youngdeungpo-Ku, 150-101 Seoul, Korea
Abstract
In this paper, described are a neural network model and system implementation for recognizing characters extracted from license plates of moving vehicles. A method for selecting features appropriate for recognition is proposed based on relationships between character features and recognition rate, and an enhanced back-propagation algorithm is also proposed which e ectively selects training patterns and dynamically modi es learning rate. Based on the proposed algorithm, a character recognition system for license plate is developed and tested against real data. In the test performed on vehicles running on the roads, the system demonstrated recognition rate higher than 95 percent.
1 Introduction Neural networks have been successfully employed in various applications of pattern recognitions. Especially, neural network has demonstrated good performance in recognizing noise-stained characters and hand-written characters, and thus neural network implementation might be appropriate for recognizing characters extracted from moving vehicles[5, 6, 7]. Real-time recognition of characters for vehicle license plates is very dicult, as the size of characters varies depending upon the position of the image extraction, motion of the camera, and speed of the vehicle. The required complexity of the system is very close to that of recognizing hand-written characters. Researches on developing neural network-based character recognition systems have mainly used features extracted based on heuristics. However, the feature extraction methods currently being used have not proved their validity through systematic analysis, and the features tend to lose its distinctive features, because of the uncertainty involved in feature extraction and overlapped features. In addition, the number of features is so large that recognition requires enormous computation time and thus makes it almost impossible to implement on current hardwares. Back-propagation has been known to be useful in training multi-layered neural networks. However, disadvantages of the algorithm are that it requires a large computational time for training, possibly converges into local minimum, and forgets previously learned weights in the process of training[3, 4, 9]. In this research, a back-propagation algorithm is employed to recognize characters in the license plate of moving vehicle in real-time. Input-node removal method is proposed as an e ective way of extracting features from the object, and an enhanced back-propagation algorithm is employed based on selective training pattern to prevent oscillation and to facilitate fast convergence. The proposed methods were successfully employed in recognizing characters in the license plate of moving vehicles.
2 Neural Network Design for Character Recognition Character recognition system based on neural networks consists of four sequential steps: preprocessing, feature extraction, training, and recognition. Preprocessing step is comprised of segmentation, noise ltering, 1
and normalization[7, 9]. Feature extraction is a process to determine how to explicitly describe the character pattern, generally by constructing feature vectors representing characters, and thus have a strong in uence on overall recognition rate. Improperly selected features frequently lead to low recognition rate and require complex recognition algorithms[1, 7, 9]. Analysis on the relationship between features and overall recognition rate in back-propagation algorithm can lead to performance improvement of the recognition system by selecting relevant features. As the features propagate its in uence up to the output nodes, a direct relationship between input features and recognition rate can be analyzed by activating neural networks, purposefully removing relevant input nodes, and then measuring recognition rate. Although an attempt has been made to measure the in uence of input features by analyzing distributions of weights in hidden layers[2], the interpretation of weight vectors is very dicult. Removal of irrelevant input nodes can be done by settingPvalues of those input nodes to zeros, which in turn sets the weighted summation of hidden nodes, net (= j x ) zero and thus lead to none contribution to output values. Accordingly, we propose that the in uence of extracted features on recognizing characters can be known by the analysis of the relationship between the removed input node and the recognition rate, thus system performance can be improved by e ective feature selection. j
i
i
Step I: Selective Learning current tss = 0.0; Loop number of input for each pattern compute actual output(); compute error(); current tss = current tss + current error; If current error > average error Then adjust weights(); end if end of loop Step II: Adapt Learning Rate average errort = current tss / number of input; delta error = average errort?1 - average errort ; tolerance = - number of output * number of input / E6; If delta error > tolerance Then oscillation = oscillation + 1; end if oscillation criterion = epoch mod number of input; If oscillation criterion = 0 Then learning rate = initial learning rate * oscillation / number of input; oscillation = 0; end if
Box-1: Enhanced Back-Propagation Algorithm In back-propagation algorithm, the dynamic modi cation of learning rate and the e ective selection of training patterns can improve system performance. The e ective selection of training patterns can be done in the following operations: divide the total error sums obtained in the forward pass by the number of training patterns to calculate an average error, and then train patterns which have error larger than average error in the backward pass. This process is described in step I in Box-1. In general, training the partial set of input patterns would lead to oscillation and it can be solved by dynamic modi cation of learning rate. Dynamic modi cation of learning rate can be implemented by counting the number of oscillation when the error size increases over the predetermined range, re ecting the ratios in setting learning rate, and eventually decreasing learning rate in case of oscillations, while increasing learning rate in case of convergence, to get the faster convergence of the training. Box-1 describes enhanced back-propagation learning algorithm. 2
In the enhanced learning algorithm, tolerance value, in addition to the initial learning rate and momentum, must be determined to measure error increase for detecting oscillation. Also, periods of modifying learning rate should be determined. In the algorithm, the learning rate should be modi ed whenever the number of epochs increased is equal to the number of training patterns, as described in step II in Box-1. Selective training patterns might reduce computation complexity in each epoch, and dynamic modi cation of learning rate might e ectively prevent oscillation frequently encountered in back-propagation algorithm. Both mechanisms might improve convergence speed and degree of generalization. In the applications of real world problems, the enhanced back-propagation algorithm e ectively solved a set of complex training patterns. In case of easy training patterns the number of epochs in the enhanced algorithm tends to become larger than in the conventional algorithm. However, the total training time in enhanced algorithm was reduced through e ective decreases of computation time each epoch.
3 Recognizing Characters in Vehicle License Plate Currently, in Korea the vehicle license plate consist of eight characters, except some special use. The rst two characters in the upper row are Hangul(Korean script) characters (geographical regions), and the third character in the upper row is a numeral (vehicle class). The rst character in the lower row is a Hangul character, while the next four characters in the lower row are four numerals as in the Figure-1. Part (a) in Figure-1 depicts a license plate in 512*480 gray-scaled vehicle image captured by CCD camera, while part (b) depicts digitized result of license plates after operations of segmentation and preprocessing for each character. The resolution ratio of preprocessed character strings, except four numerals of relatively large size in the lower row, is very low, which imposes complexities and inherent diculties on pattern recognition. In the review of 200 images of license plates in the experiment it was found that the size of Hangul characters in the upper row varies from 13*15 to 27*29 pixels, and the size of Hangul characters in lower row takes various ranges of 16*23 to 38*47 pixels. The size of numerals varies too; numerals in the upper row change the size from 8*17 to 20*24 pixels, while numerals in lower row change from 10*49 to 29*60 pixels.
(a) License Plate in Vehicle Image (b) Segmentation and Digitized Figure 1: Korean Vechicle License Plate The rst two Hangul characters in the upper row indicate geographical region, one of the six registration cities and nine registration provinces. Five out of the six types of Hangul syllable[8] are included in the composition of the two characters. The resolution of two characters' image is so low that it is very dicult to separate vowels and consonants, and to extract features via Bar Masking. When combining two characters into a complete pattern after separately recognizing individual character was attempted, the uncertainty exponentially increases. Therefore, method of simultaneous recognition of two characters as a single pattern was employed. The rst Hangul character in the lower row has the structure of "consonant + vowel", generating 84 di erent character patterns. In the Hangul character recognition, a single character is divided into vowel and consonant. A structural method was employed for vowel recognition, while neural network was employed for consonant recognition. The vertical consonants of Hangul characters are written in the simpli ed form but horizontal consonants are written in the cursive style. It is advisable to divide consonants 3
into horizontal consonants and vertical consonants, and to recognize each of them. Numerals are comprised of 10 di erent patterns from 0 to 9, and it is more e ective to divide numerals into small size numerals and large size numerals and recognize each of them. Figure-2 depicts Hangul characters and numerals which are used in Korean vehicle license plates.
Figure 2: Characters used in Korean Vehicle License Plates In this research of neural network application, constructed are ve neural networks, each for: geographical regions, small size numerals, large size numerals, vertical consonants, and horizontal consonants of Hangul characters. Out of 300 license plates extracted from vehicles running on the roads, 200 license plates were used for training and another 100 license plates were used for testing recognition rate. For the purpose of network architecture, the number of input nodes was determined by the features, the number of hidden nodes was by experience, and the number of output nodes was by the number of output codes.
4 Experiments and Results In the experiments, training was done until recognition of training patterns was 100 percent correct, and then variations of recognition rate with removals of input nodes was investigated. Investigation on the geographical region codes showed that 15 features out of 34 features, when they were removed, did not have a strong impact on recognition rate, generally less than 1 percent. In particular, 5 features had completely no in uence on the recognition rate. However, when all of the 15 features were removed simultaneously, the recognition rate was lowered to 59 percent and therefore it was revealed that the overall recognition rate was greatly in uenced by the removal of irrelevant nodes. The recognition rate was easily recovered by adjusting weights with a small number of iterations. With the respect of feature selection, only small number of training patterns can e ectively determine features of trainable patterns. Table-I shows the architecture of Table 1: Experimental Results of Characters Recognition Characters Geographical Region Small Size Numberal V ertical Consonant Horizontal Consonant Large Size Numeral
NeuralNets Experimental Data Feature Selection Topology Numberof Number Number Numberof Numberof Recog: (input hidden Training of Test of Removale Determined Rate(%) output) Patterns Patterns Features Features Features 29 * 29 * 15
200
100
34
15
29
96.0%
25 * 25 * 10
200
100
37
15
25
95.0%
14 * 14 * 14
152
78
16
6
14
94.9%
14 * 14 * 14
48
22
16
7
14
90.9%
25 * 25 * 10
800
400
37
14
25
98.3%
neural network employed in the experiment, number of patterns, the number of features for each character, and test results of characters recognition of vehicle license plates. 4
In the Table-1, the number of removable features, having no impact on overall recognition rate even with the removal of the features, was determined in the beginning step considering that even simultaneous removal of some features would not have any impact on recognition rate. In case of numeral recognition, relatively many features could be removed without a signi cant impact. However, in case of Hangul characters and regional codes, because of the low resolution, all the features showed relevant to the overall recognition rate. In this research, the enhanced back-propagation algorithm was employed for neural network application. The algorithm was useful and e ective for overcome local minima reached possibly when the regional code "(Kyunggi)" was trained "(Kyungbuk)" for similar character pattern, and for speeding up the convergence. The newly developed algorithm demonstrated much improved performance level when applied to complex and dicult patterns, but showed almost equal performance to conventional algorithms when applied to easy patterns. It was found that the tolerance value used for determining error increase to detect oscillations is very sensitive to learning speed. The degree of generalization was much enhanced and better results were demonstrated in the experiment when the training patterns were uniformly distributed and the size was large.
5 Conclusion In this paper, an enhanced back-propagation algorithm and feature selection method are described for improving convergence speed and were employed in an character recognition systems of license plates extracted from moving vehicles. The proposed algorithm can enhance system performance through e ective feature extraction, reduction of recognition time and learning speed, and increased recognition rate. The vehicle license plates recognition system implemented in T800 Transputer-based environment[10] showed that the time for a complete recognition required only 0.09-0.11 seconds after digitized and segmentation, which was not problematic in real world applications. The recognition rate, even though varied depending on the results of preprocessing step, was generally above 95 percent, relatively high performance. This research requires further experiments on training more samples, especially needs further research on new feature extraction methods and improving the degree of generalization.
References
[1] A. Rajavelu, M. T. Musavi, and M. V. Shirvaikar, \A Neural Network Approach to Character Recognition," Neural Networks, Vol.2, pp.387-393, 1989. [2] Caudill M., \Using Neural Nets: Representating Knowledge," AI Expert, pp.34-41, Dec. 1989. [3] Don R. Hush and Bill G. Horne, \Progress in Supervised Neural Networks," IEEE Signal Processing Magazine, pp.8-39, Jan. 1993. [4] David E. Rumelhart, Geoo ney E. Hinton, and R. I. Willians, \Learning Internal Representations by Error Propagation," in David E. Rumelhart, James L. McClelland, and PDP Research Group (Eds.), Parallel Distributed Processing, Vol.1, pp.318-362, MIT Press, 1986. [5] F. Lisa, J. Carrabina, C. Perez-Vicente, N. Avellana, and E. Valderrama, \Two-bit Weights Are Enough To Solve Vehicle License Recognition Problem," in Proceedings of International Conference on Neural Networks, San Francisco, Vol.III, pp.1242-1246, Apr. 1993. [6] Luisa DE VENA, \Number Plate Recognition by Hierarchical Neural Networks," in Proceedings of 1993 International Joint Conference on Neural Networks, Nagoya, Vol.III, pp.2105-2108, Oct. 1993. [7] Robert Schalko , Pattern Recognition : Statistical, Structural and Neural Approaches, John Wiley & Sons, Inc., 1992. [8] Sung-Bae Cho and Jin H. Kim, \Recognition of Large-Set Printed Hangul (Korean Script) by Two-Stage Backpropagation Neural Classi er," Pattern Recognition, Vol.25, No.11, pp.1353-1360, Nov. 1992. [9] Pao Yoh-Han, Adaptive Pattern Recognition and Neural Networks, (Reading, Mass., Addison-Wesley Publishing Company, 1989). [10] INMOS Ltd, The Transputer Data Book, inmos, 1989.
5
