2011 Second International Conference on Emerging Applications of Information Technology
I H Isolated Handwritte en Devnaggri numerral recogniition usingg HMM Sanndeep B. Patill
Vaisshali S. Patil
Associatee Professor (ETT&T), Shri Shankaracharya Collegee Of Engg. & Tech., Bhilai E Email: [email protected]
Professsor & Head (ITT), Shri Shankaracharya Collegee of Engg. & Tech., T Bhilai Email: [email protected]
Assistantt Professor (ET& &T), S Shankarachhrya Institute of Shri o Tech. & Manaagement, Bhilai Email: [email protected]
Devnnagri characteer but none of o paper was found on Devnnagri numeral recognition using u HMM. That’s T why we have h been motivvated to proceeed our work onn Devnagri numeeral recognitionn. “Fig.1” show ws handwrittenn Devnagri numeerals from threee different persons.
A Abstract:—This paper describes a complete system for thee r recognition of issolated handwritten Devnagri numerals usingg H Hidden-Markov v model (HMM M). The HMM has the propertyy that its states are a not defined as a priory in nformation, butt a are determined d automaticallly based on a database off h handwritten nu umerals imagees. In this woork the imagee d database consisst of 500 imag ges of handwrritten Devnagrii c characters from m 50 different writers. w Beforee extracting thee f features, the im mages are norm malized using im mage isometricss s such as translaation, rotation and scaling. An automaticc s system trained 400 images of image databasse and numerall m model form wiith multivariatte Gaussian sttate conditionall d distribution. A separate s set of 100 1 characters was w used to testt the system. The T recognitio on accuracy for individuall n numerals variess from 30% to o 100% for N= =3 and 80% too 1 100% for N=5.
Figgure 1: Devnagri numerals.
K Keywords: Devn nagri; HMM; Multivariate; Gaussian; mu;; s sigma ;
T binary imaage written inn MS-paint connsists of a The blackk foreground in i front of a large white background b Hencce the image is inverted suchh that the backkground is blackk and the forreground is white. w This iss done by subtrracting the binnary image froom a matrix off 1s of the samee size. Moreover, smaller num mber 1s will mean m lesser calcuulations in corrrelation. To exxtract features, which are invarriant to transllation and scaaling, it is neecessary to norm malize images. For the proceess of normaliization the methhod of momeent normalizattion is used, which is propoosed by Paraantonis and Lisboa L . Thhe regular geom mentrical mom ment of order zero z and one is used to find the t centre of gravity g of centrroid. The ( p + q) th order geom metrical moment  of a diigital image f ( x1 , y1 ) of size MxM is givenn as
I. INTRODU UCTION The process of handw writing recogniition involvess e extraction of soome defined characteristics c called featuress too classify an unknown u hand dwritten characcter into one off thhe known claasses. A typiccal handwritinng recognitionn s system consistss of several stteps, namely: preprocessing, s segmentation, feature extracction, and classsification . S Several types of decision methods, m includding statisticall m methods, neuraal networks, sttructural matchhing (on trees, c chains, etc.) The T stochastic processing (M Markov chains, e etc.) have beenn used along wiith different tyypes of featuress [2-4]. Many reecent approach hes combine seeveral of thesee techniques together in order to obtaain improvedd r reliability, deespite wide variation in handwriting. D Devnagri is onne of the official languagess in India andd w which is used by b majority off peoples. It beelongs to groupp o language aloong with Maraathi, Hindi, Bengali, Gujrathii of a other northh Indian languaages . and
M −1 M −1
M p,q = ∑
∑ x1 y1 f ( x1 , y1 ).
x1 = 0 y1 = 0
T zeroth ordeer moment cann be obtained by putting The p = q = 0 in (1). Itt will be then trransformed into
II. DEVNAG GRI SCRIPT The Devnaagari script fo ollows left to right fashion n f writing. Th for he Devnagari alphabet is ussed for writing g H Hindi, Sansk krit, Marathi,, Nepali laanguages . E Each Devnagaari consonant has h an inhereent vowel (A). V Vowels can be written as ind dependent letteers, or by usingg a variety of diiacritical mark ks which are written w above, b below, before or after the consonant thhey belong too D Devnagri scriptt. A lot of reseearch work hass been done onn 978-0-7695-4329-1/11 $26.00 © 2011 IEEE DOI 10.1109/EAIT.2011.10
I IMAGE NORM MALIZATION N AND F FEATURE EX XTRACTION
M −1 M −1
M 0,0 = ∑ ∑ x1 0 y1 0 f ( x1 , y1 ). x1 = 0 y1 = 0
W can get first order momennts in two wayys either by We puttinng p = 1 andd q = 0 or p = 0 and q = 1 . With p = 1 and q = 0 wee get first orderr moment as
M −1 M −1
M 1,0 = ∑ ∑ x11 y1 0 f ( x1 , y1 ). x1 =0 y1 = 0
A. Numeral model formation with multivariate Gaussian state conditional distribution:
A HMM with multivariate Gaussian state conditional distribution consists of  Pi0= Row vector containing the probability distribution for the first unobserved state π0(i) = P (s1 = i) (5) A= Transition matrix: aij=P (st+1 = j׀st=i) (6) mu= Mean vector stacked as row vector such that mu(i, :) is the mean vector corresponding to i-th states of the HMM. Sigma= Covariance matrix. These are stored on one above the other in two different ways depending upon whether full or diagonal covariance matrices are used. For diagonal covariance matrices, sigma (i , :) contain the diagonal of the covariance matrix for the i-th state. For the purpose of isolated handwritten numeral recognition, it is useful to consider left- to- right models. In left-to-right model transition from state i to state j is only allowed if j ≥ i, resulting in smaller number of transition probabilities to be learned. The clusters of observation are created for each model separately by estimating Gaussian mixture parameter for each model. The function mu and sigma able to determine the dimension of the model and the type of covariance matrices i.e. size of the observation vector and the number of states. The matching process computes a matching score between the sequence of observation vector and each numeral model using the Viterbi algorithm . After post processing, a lexicon sorted by matching score is the final output of the numeral recognition system. For transition matrix A, the row vector summation must be equal to 1 for any number of states N. The transition matrices (3x3) for N=3 shows in Table 1, and (5x5) for N=5.
From first order moment we get important information about the location of object in the image. It is called as center of gravity or centroid. If it is assumed to be ( x , y ) , then centroid of the object is calculated as .
M 1,0 M 0, 0
and y =
M 0,1 M 0, 0
In the database objects in the images can have centroid anywhere in the image frame. But we want to translate it to the center of the image frame (33,33) of image size 65X65. The “Fig.2” shows the original image for numeral zero and image after translation (33,33) and scaling by factor β=400.
Original Image (b) Image after translation and scaling Figure 2: Image Normalization.
The features that are extracted from the images are the (dimension) height and the width of the observed vector by using mu and sigma function shown in “Fig. 3”.
Figure 3: Height and Width of the numeral zero.
TABLE 1: TRANSITION MATRIX.
0.85 0.0 0.0
A hidden Markov model is a doubly stochastic process, with an underlying stochastic process that is not observable (hence the word hidden), but can be observed through another stochastic process that produces the sequence of observations [2, 4, 6-7]. The hidden process consists of a set of states connected to each other by transitions with probabilities, while the observed process consists of a set of outputs or observations, each of which may be emitted by each state according to some output probability density function (PDF) [8-12]. Depending on the nature of this PDF function several kinds of HMMs can be distinguished.
0.15 0.85 0.0
0.0 0.15 1.0
The N-mean vector and the covariance matrices  for N=3 is (3X5) matrix and for N=5 is (5X10) matrix. The Nmean vector and the covariance matrices for numeral zero to nine are calculated for N=3 as shown in Table 2 to 11.
Figure 4: Numeral ‘Zero’ in Devnagri script.
TABLE 2: N-MEAN AND DIAGONAL OF COVARIANCE MATRIX OF ZERO
TABLE 5: N-MEAN AND DIAGONAL OF COVARIANCE MATRIX OF THREE.
N-mean vector of numeral Zero -30.6121 5.5032 0.2297 1.8466 -1.0598 -39.8968 3.2858 0.8861 0.3554 -0.2421 -48.9952 3.7816 0.3203 0.5388 -0.2641 Diagonal of covariance matrices for Numeral ‘Zero’ 4.8728 0.1395 0.0348 0.0055 0.0024
N-mean vector of numeral ‘Three’ -33.514 2.8048 2.602 1.562 0.9476 -44.9259 4.2216 0.4562 0.2547 -0.0529 -55.9676 6.996 3.5299 -3.2973 0.6361 Diagonal of covariance matrix for numeral ‘Three’ 7.0643
Figure 5: Numeral ‘One’ in Devnagri script.
Figure 8: Numeral ‘Four’ in Devnagri script.
TABLE 3: N-MEAN AND DIAGONAL OF COVARIANCE MATRIX OF ONE.
TABLE 6: N-MEAN AND DIAGONAL OF COVARIANCE MATRIX OF FOUR.
N-mean vector of numeral one -36.2465 5.8923 2.415 -1.8379 0.7062 -45.8329 2.9168 0.7458 0.3669 0.1767 -55.5438 2.1111 3.5814 0.4625 1.1755 Diagonal of covariance matrix for numeral ‘One’ 20.4049 1.3622 1.4973 0.1392 0.1855
N-mean vector of numeral ‘Four’ -27.4733 7.0322 -3.0406 3.2624 -1.4921 43.3635 4.2036 0.2905 0.3599 -0.3307 -60.6548 7.8149 -0.8336 0.7484 -0.5853 Diagonal of covariance matrix for numeral ‘Four’ 8.5489
Figure 6: Numeral ‘Two’ in Devnagri script. TABLE 4: N-MEAN AND DIAGONAL OF COVARIANCE MATRIX OF TWO.
Figure 9: Numeral ‘Five’ in Devnagri script.
N mean vector of numeral ‘Two’. -19.4804 2.4906 0.2141 0.4511 0.5103 -38.8845 3.2658 2.6757 -0.1781 -1.0144 -46.1308 1.8772 1.5549 0.5203 -0.2213 Diagonal of covariance matrix for numeral ‘Two’ 7.9464
TABLE 7: N-MEAN AND DIAGONAL OF COVARIANCE MATRIX OF FIVE.
N-mean vector of numeral ‘Five’. -37.4011 4.8927 0.2072 -0.0174 1.468 -38.5795 2.2104 1.9201 0.6765 -0.285 51.8501 7.4937 -1.8673 0.9182 -0.5587 Diagonal of covariance matrix for numeral ‘Five’ 18.0842 1.6406 0.4402 0.3682 0.1556
Figure 7: Numeral ‘Three’ in Devnagri script. Figure 10: Numeral ‘Six’ in Devnagri script.
TABLE 8: N-MEAN AND DIAGONAL OF COVARIANCE MATRIX OF SIX.
TABLE 11: N-MEAN AND DIAGONAL OF COVARIANCE MATRIX OF NINE.
N-mean vector of numeral ‘Six’. -33.948 3.838 1.9046 1.2439 -0.9181 -44.8144 3.6197 0.5164 0.032 0.0045 -52.3264 4.7887 0.1398 1.6045 -1.3352 Diagonal of covariance matrix for numeral ‘Six’
N mean vector of numeral ‘nine’
-47.4194 6.5576 -1.4081 2.222 -1.4028 -48.6214 3.2756 1.7443 0.1729 -0.5684 Diagonal of covariance matrix for numeral ‘Nine’
TABLE 9: N-MEAN AND DIAGONAL OF COVARIANCE MATRIX OF SEVEN.
N-mean vector of numeral ‘Seven’ -33.047 2.1511 1.9602 1.7392 0.3396 -52.8563 3.1718 4.854 -1.0383 -1.6435 -51.0085 4.0763 0.0762 0.5039 -0.2456 Diagonal of covariance matrix for numeral ‘Seven’ 18.7937 1.8074 1.0087 0.6771 0.3089
Fig. 12: Numeral ‘Eight’ in Devnagri script. TABLE 10: N-MEAN AND DIAGONAL OF COVARIANCE MATRIX OF EIGHT.
N-mean vector of numeral eight -27.928 4.248 0.7411 0.1983 0.6817 -41.5983 4.628 0.2717 0.5965 -0.4783 -53.8643 9.7178 -1.7841 0.5004 0.299 Diagonal of covariance matrix for numeral ‘Eight 5.0092
As there does not exist any standard database of character images. In the present work the image database is collected from 50 different persons, starting from school students to Doctorates. It consists of 500 digital images of size 65X65. Initially, the handwritten digits are collected on plain paper (non-ruling pages) and latter they are scanned. To make the size of images in the database constant, these scanned images are then edited with the help of Microsoft paint, available in Windows operating system. They are in arbitrary translation, rotation and scale. Out of these 500 digital images 400 images are used for training purpose and remaining 100 images are used for testing purpose. When we applied first 100 images ten samples of each numeral then the recognition result for different numerals for N=3 and N=5 is shown in Table 12. It has been observed that the numeral model values for numeral 3,6 and 1,9 has approximately same values for N=3 and hence the recognition rate decrease for that values. The recognition result can be improved by considering N=5. A Graphical User Interface for numeral Zero is shown in “Fig. 14”.
Figure 11: Numeral ‘Seven’ in Devnagri script.
Figure 14: GUI for Devnagri Numeral Zero. Figure 13. Numeral ‘Nine’ in devnagri script.
TABLE 12: THE RECOGNITION ACCURACY FOR DEVNAGRI NUMERALS.
1. Applied Devnagri Numerals
Numerals recognized properly
Numerals recognized properly
S.J. Parantonis and P.J.G. Lisboa, “Translation, rotation and scale invariant pattern recognition by high-order neural network and moment classifiers”, IEEE Trans. Neural networks, vol.3, No.2, pp.241251, 1992. 2. “H2M: A set of MATLAB/OCTAVE functions for the EM estimation of mixture and Hidden Morkov Model” by Olivier Cappe ENST. Dpt. TSI/LTCI (CNRS-URA 820), France, August 24, 2001. 3. Rafael C. Gonzalez and Richard E. Woods, “Digital Image Processing”, Addison-Wesley publishing company 4. Magdi A. Mohamed, and Paul Gader, “Generalized Hidden Markov Models-Part II: Application to Handwritten Word Recognition”, IEEE Trans. On Fuzzy System, Vol. 8. No.1, February 2000 5. S. K. Parui and B. Shaw. ,”Offline handwritten devanagari word recognition: An hmm based approach.”, Proc. PReMI-2007(Springer), LNCS-4815:528-535, December 2007. 6. L. R. Rabiner., “ A tutorial on hidden markov models and selected application in speech recognition”, Proceedings of the IEEE. 77(2):257-286, February 1989. 7. Christopher M. Bishop, “Pattern Recognition and Machine learning”, Information science and statistic series, Springer. Pg. 423-455. 8. Jia Zeng and Zhi-Qiang Liu, “Markov Random FieldBased Statistical Character Structure Modeling for Handwritten Chinese Character Recognition”, IEEE Trans. On Pattern analysis and Machine aintelligence, Vol. 30, No.5, May(2008). 9. Stephan R. Veltman and Ramjee Prasad, “Hidden Morkov Models Applied to On-Line Handwritten Isolated Character Recognition”, IEEE Trans. On Image Processing, Vol.3, No.3, (1994). 10. Thierry Artie, Sanparith Marukatat and Patrick Gallinari, “Online Handwritten Shape Recognition Using Segmental Hidden Markov Models”, IEEE Trans. On Pattern Analysis and Machine Intelligence, Vol. 29, No.2, February (2007). 11. A. Vlontzos and S.Y. Kung, “Hidden Morkov Model for character recognition”, IEEE Trans. On Image processing, vol. 1, No. 4, pp. 539-543, October 1992. 12. Gernot A. Fink, “Morkov model for pattern recognition from theory to application”, Springer publication-2003.
In this present work we have proposed an HMM based approach for recognition of isolated handwritten Devnagri numerals. The recognition result obtained from this work varies for numeral to numeral for number of states varies from 3 to 5. Our further work will be to improve the recognition result up to 100 % for each numeral, as well as for Devnagri word recognition. For this purpose we can used the combined method i.e. both (Analytical & Holistic) which can reduce the drawback of this method and have the advantage of combined method.