Fusing Cortex Transform and Intensity based Features for Image Texture Classi cation Md. Khayrul Bashar
Noboru Ohnishi
Dept. of Information Engineering
Dept. of Information Engineering
Nagoya University
Nagoya University
Furo-cho, Chikusa-ku,Nagoya, Japan.
Furo-cho, Chikusa-ku,Nagoya, Japan
[email protected]
[email protected]
This paper proposes a new scheme of fusing cortex transform and brightness based features obtained by local windowing operation. Energy features are obtained by applying popular cortex transform technique within a sliding window rather than the conventional way, while we de ne three features namely directional surface density(DSD), normalised sharpness index(NSI), and normalized frequency index(NFI) as mesures for pixel brightness variation. Fusion by simply vector tagging as well as by correlation is performed in the feature space and then classi cation is done using minimum distance classi er on the fused vectors. It is interesting that the brightness features, though inferior on some natural images, often produces smoother texture boundary in mosaic images, whereas energy features show the opposite behavior.This symmetrically inverse property is combined through vector fusion for robust classi cation of muti-texture images obtained from Brodatz album and VisTex database. Classi cation outcome with confusion matrix analysis shows the robustness of the scheme. Abstract {
by some authors[1] [2]. Recently Huang et al.[3] used consensus theory to integrate deterministic and indeterministic elds of texture. In this study, we propose to integrating features derived from multi-resolutional Cortex Transform, proposed by Goresnic et al.[5]and with some intensity based features. Though fractal features are quite e ective for texture analysis, it is tedius to compute them precisely. Hence we de ne three simple intensity features namely normalized sharpness index(NSI), normalized frequency index(NFI) and directional surface density(DSD)as alternative to fractal dimension. The following are the characteristics of our approach. 1. Three simple measures, namely DSD, NSI and NFI, are formulated to characterize pixel brightness variations. 2. In Cortex transform approach, the processing is block-based like short space Fourier transform, where the block size is xed experimentaly. Details will be found in the reference [13]
Cortex transform, texture, mean energy, directional surface density, integration, Confusion matrix. Keywords:
3. Block operation, where block-size and octave scale control the total number of lters i.e., features space, eliminates two steps namely the lter selection and post- ltering transformation as in[4].
1 Introduction
Texture classi cation is very important in image analysis. Content based image retrieval, inspection of surfaces, object recognition by texture, OCR are examples where texture classi cation plays a major role. Multichannel ltering methods[4][5][6][8][7][9] o er potentiality and computational advantages over other methods for texture classi cation and segmentation. However, all the methods are highly dependent on the types of images or the imaging sensors. The particular class of techniques suitable to images from one sensor type fails for the other sensors. Also all aspects of textures are not captured by any single class of descriptors or mechanisms. Combining texture features has been suggested
ISIF © 2002
4. Our fusion approach combines the goodness of brightness and transform features to produce more robust classi cation. Section 2 brie y mentions the Cortex Transform with feature computation technique and section 3 explains the proposed brightness variability features. Section 4 describes feature integration and classi cation, and section 5 reveals an experimental study with classi er performance analysis. Lastly conclusion and future expansion of this work are discussed.
1463
2 Cortex Transform
where Fij (x; y) is the ltered image, and F is its mean in ith radial band and j th orientation. A simple The cortex transform [5] decomposes an input image feature computation procedure is given below. into a set of subimages according to the behaviour of simple cells in the human visual system(HVS). The band pass nature of the simple cells is modelled by the Cortex 1. Choose a square block(16x16) of data, compute dft lter, which is a product of two Gaussian functions in magnitude and translate the transform origin to the the frequency domain, namely Radial band and Oriencenter of the block. tation lters with a radial and orientation bandwidth of 1 octave and 45 degrees respectively. In polar cordinate 2. Compute the number of lters within the block and system, they are represented as follows. construct their kernels using Eq. 1 to Eq.3. Radial band lter: 3. Multiply the transformed data with each of the ker1 expf (r mr )2 g nels, compute features using Eq.4, 5 and store them f (r) = p (1) 2r 2 2r into seperate arrays. Orientation lter: 4. Shift the window horizontally and vertically by ap1 ( m )2 propriate pixels(one pixel in our implementation) (2) g () = p and follow steps 1 to 3 untill the whole image is 2 expf 22 g scanned. This generates a set of feature images. Cortex lter: C (r; ) = f (r)g () (3) 3 Features Based on Pixel Brightness Variation The frequency spectrum of a sample radial band , orientation, and cortex lter are shown in Fig.1. The detail 3.1 Feature De nition computation of various lter parameters with lter kernel generation using Eq.(1) to (3) are given in [5] [13]. It has been reported that fractal features are useful Only the feature image generation scheme is explained for texture analysis [10] [11] [12]. However, precise values of these features are diÆcult to compute especially here for illustrating its purpose. for a pattern of small area. Hence we attempted to de ne the following features based on the pixel brightness variation. Directional Surface Density(DSD) is de ned as: 1 MX1 NX1 jf (x; y) DSDx;y (x; y ) = MN x=0 y =0 Figure 1: The frequency spectrum of a sample (a) Radial f (x + x; y + y )j (6) band lter, (b) Orientation lter, and (c) Cortex lter where x = 1; 2; 3; ::::::::; M: 2.1 Feature Image Generation y = 1; 2; 3; ::::::::; N: The generation of feature images is done by sliding block operation, where the block or window size is de- and M, N are the height and width of an window termined based on a boundary veri cation experiment of an intensity image f (x; y) centered at (x,y). This as explained in [13]. feature corresponds to the directional surface value of gray scale image when it is considered as the three We compute texture features based on the avgerage the dimensional constructed by x, y, and brighness energy and square of magnitude deviation over the l- axes. It canobject be computed for many directions and tered images as follows : spacing of interest by vaying the values of x and y and found to be e ective in texture feature extraction N X1 NX1 with noise suppression and boundary preservation. 1 Eij (x; y ) = j Fij (x; y )j2 ; (4) Normalized Sharpness Index(NSI)is de ned as: 2 N ( ) = N12
2 x; y M Dij
=0 y=0 X1 NX1
ij
x
N
=0 y=0
x
(jFij (x; y)
Fij
)j2 (5) 1464
( ) =
N SI x; y
1
areasize
X i
SBCi
(7)
i 2 ; where SBCi = 10 ifelsewhere Here = f Sk g, is a set of all discrete points, where a condition f (Sk 1) f (Sk ) > th is satis ed. Sk , a smallestPpositive integer beyond P0Sk 1 , can be de ned as k Sk = n , where S = i 0 i=1 i=1 ni = 0, ni 's are the spatial co-ordinate values at the satisfactory points, and f(.) is a 1D version of the image. This feature indicates the sharpness in the brightness changes. However, proper threshold selection is important. We used 15 % of the dynamic range of the image as threshold value. Normalized frequency index(NFI)is de ned as:
\dissimilar", and \as it is" groups based on the threshold (0.90,0.25) of the feature correlation coeÆcient values computed as follows. r
= pSSf pf S m n
fm
fn
;
(9)
where Sf , Sf , and Sf f are variances and covariance of the mth and nth (where m 6= n) feature imagesfm and fn respectively. Then we used 'AND' and 'OR' logic among the similar and dissimilar features to obtain the fused features. The fusion rules are as follows: X 1 Rule 1: if r 0:90, features are similar. So the fused N F I (x; y ) = F BCi ; (8) feature = MIN(feature1, feature2, ............, featureN). areasize i Rule 2: if r 0:25, features are dissimilar. So the fused 8 feature= MAX(feature1, feature2,......,featureN). < 1 if i = Sk 1 2 ; However we left the rest of the features in \as it is" group where F BCi = : and DIF Fk DIF Fk+1 < 0; untouched and directly applied to the classi er. 0 elsewhere P k 1 Here DIF Fk = f (Sk 1 ) f (Sk ) or f ( i=1 ni ) 4.2 Classi cation P k f ( i=1 ni ) and k = 1; 2; :::; kmax Classi cation of the fused features is performed in supervised manner using minimum distance classi er that This feature indicates the frequency of the brightness uses Euclidean as similarity measure. In the changes. This is also an e ective feature for texture training phase, distance we select some samples from each texcharacterization for an appropriate threshold value. ture class of the input image by visual inspection. Then all the features(cortex and brightness) are computed as 3.2 Feature Computation explained in section 2 and 3. In order to obtain repreThe above brightness features are computed directly sentative vectors, we just average out the feature from a rectangular sliding window centered around each vectors ofclass the samples each class and stack them pixel of an image. It is important to use larger win- together. The training from vectors formed and the fused dows(area less than the smallest texture class) to grow feature vector, corresponding tosoeach co-ordinate, the texture features more reliably and to remove un- are inputted to the classi er. Classi erimage iteratively labels necessary noises. However, larger window oversmooths each pixel, based on the minimum distance criterion. various texture classes and produces double boundary Two sample feature vectors for Cortex transform and among the inter-texture classes. To minimize the e ect, brightness based features are as follows: we used an square window size of 13 to 17 for the images of interest. Vc (x; y ) = [Eij (x; y ); M Dij (x; y )] 4 Integration and Classi cation Vb (x; y) = [N SI (x; y); N F I (x; y); DSDxy (x; y)] where i = 1; 2; 3; ::::; S: 4.1 Integration j = 1; 2; :::; dmax : In our approach, we used feature level integration x = 1; 2; ::::::; R: through combining various candidate features into two y = 1; 2; :::::::; C: ways namely vector fusion and correlation co-eÆcient based fusion guided by fuzzy logic literature as in[14] S is total scales and dmax indicates total orientaIn the vector fusion method, we just compute feature Here tions. vectors from di erent approaches, for example cortex transform and brightness measures. We then put them sequentially to obtain a fused vector at each spatial 5 Experimental Results point. Fused vector corresponding to each pixel is com- We conducted an experiment on the two groups of impared with the class representative fused vectors to ob- ages obtained from the standard Brodatz texture album tain classi ed images. and also from VisTex database in the internet. Twenty In the second approach, we are motivated by fuzzy logic, mosaic images from Brodatz album and thirty natural where we devided the derived features into \similar", images from VisTex, all having (256x256) pixels, were 1465
m
n
m n
Figure 3: (a) Original mos31 image, Results of(b)Cortex transform features,(c) contrast features, Histogram of(d)the average ltered energy,(e)the magnitude deviation of ltered image, and (f) NSI, NFI, DSD features.
(a)
(b) Average energy histogram for scene2 image 0.09
AE ch-1 AE ch-2 AE ch-3 AE ch-4 AE ch-5 AE ch-6
Normalized histogram
0.08 0.07 0.06 0.05 0.04 0.03 0.02 0.01 0 0
5
(c) 0.1
0.12
45
50
45
50
DSD-H DSD-V DSD-1D DSD-2D
0.1 Normalized histogram
Normalized histogram
0.07
40
DSD histogram for scene2 image
Magnitude deviation ch-1 Magnitude deviation ch-2 Magnitude deviation ch-3 Magnitude deviation ch-4 Magnitude deviation ch-5 Magnitude deviation ch-6
0.08
15 20 25 30 35 Average filtered energy
(d)
Magnitude deviation histogram for scene2 image 0.09
10
0.06 0.05 0.04 0.03 0.02
0.08 0.06 0.04 0.02
0.01 0
0 0
5
10 15 20 25 30 35 40 45 Standard deviation of filtered magnitude
50
0
5
10 15 20 25 30 35 40 Directional surface density(DSD)
(e) (f) 4: (a) Original scene2 image, Results of(b) CorFigure 2: Some samples of Brodatz mosaic-texture and Figure tex transform features,(c) contrast features. Histogram VisTex natural scene images of(d) the average ltered energy,(e) the magnitude deviation of ltered image, and (f) DSD- features.
(a)
examined in the experiment. Some of the texture images used are shown in Fig.2.
(b) 0.12
Normalized frequency
5.1 Results
Average energy, ch-1 Average energy, ch-2 Average energy, ch-3 Average energy, ch-4 Average energy, ch-5 Average energy, ch-6
0.1 0.08 0.06 0.04 0.02 0 0
(c) 0.12
15
20 25 30 35 Average Energy
40
45
50
(d) 0.09
NSI NFI DSD-H DSD-V DSD-1D DSD-2D
0.08 Normalized frequency
Normalized frequency
0.08
10
Mosaic_b image(256x256)
Magnitude deviation, ch-1 Magnitude deviation, ch-2 Magnitude deviation, ch-3 Magnitude deviation, ch-4 Magnitude deviation, ch-5 Magnitude deviation, ch-6
0.1
5
0.06 0.04 0.02
0.07 0.06 0.05 0.04 0.03 0.02 0.01
0 0
5
10 15 20 25 30 35 40 Stanadard deviation of magnitude
(e)
45
50
0 0 5 10 15 20 25 30 35 40 45 50 Normalized sharpness and frequency index(NSI,NFI), Directional surface density(DSD)
(f)
We performed classi cation using Cortex transform based and pixel contrast features as explained in section 2 and 3. In the rst method, two scales with 16x16 block, and fint of 2 units are used for a suitable ratio parameter, Rr or R between 0.01 to 0.03. Figs. 3 and 4 show two results with feature histograms(50 bins). The multiple peaks in histograms show the feature strength. However, class boundaries are not smooth enough in case of Brodatz images. On the otherhand, contrast features namely DSD, NSI, and NFI, produces smoother boundary in mosaic image as shown in Fig. 3(c), despite inferior performance to natural images especially low frequency-dominant image as in Fig.4(c). Why the cortex features(AE,MD) are successful in 1466
(a)
(b)
(a)
(b)
(c) (d) (c) (d) Figure 5: Results before and after vector integration.(a) Figure 8: Results of vector integration.(a) OrigiOriginal mos41 image, Classi ed images of(b)Cortex,(c) nal mos32 image, Classi ed images of(b)Cortex transbrightness, and (d) combined features. form,(c)intensity, and (d) combined features.
(a)
(a)
(b)
(b)
(c) (d) (c) (d) Figure 9: Results of vector integration.(a) Original Figure 6: Results before and after vector integration.(a) mos33 image, Classi ed images of (b)Cortex transform, Original scene1 image, Classi ed images of (b) cortex,(c) (c) intensity based, and (d)combined features. brightness, and (d) combined features.
(a)
(a)
(b)
(b)
(c) (d) (c) (d) Figure 7: Results before and after vector integration.(a) Figure 10: Results of vector integration.(a) Original Original scene2 image, Classi ed images of (b) cortex,(c) ower1 image, Classi ed images of (a) Cortex, (b) inbrightness, and (d) combined features. tensity based, and (c)combined features. 1467
(a)
(b)
(c) (d) Figure 11: Visual comparison of classi ed mos31 and mos41 images by our integration methods (a)and(c)stacked vector,(b)and(d)correlation method
Table 1: Comparison of overall accuracy among cortex, brightness, and fused approaches Images Sample Overall accuracy (%) pixels Cortex bright Integ -ness -rated mos21 2304 96.3 97.3 99.2 mos22 2304 97.2 97.0 100.0 mos23 2304 95.8 96.5 98.5 mos31 2304 97.4 97.3 99.2 mos32 2304 98.5 97.4 98.9 mos33 2304 95.5 97.4 97.7 mos41 3072 94.8 95.8 97.5 mos42 3072 95.3 96.5 97.8 mos51 2560 97.6 64.6 99.6 mos52 2560 95.0 75.7 98.3 scene1 2560 92.0 87.6 95.2 scene2 2560 93.2 53.6 98.4 scene3 2560 92.3 70.6 97.1 cloud1 1536 96.3 68.4 98.0 cluod2 1536 94.8 73.2 98.6 Flower1 1536 98.9 50.2 100.0 Flower2 1536 98.6 58.3 100.0 Building1 1536 91.8 77.6 94.1 Building2 1536 89.8 73.6 93.0 Building3 1536 88.6 68.3 93.2
case of natural image is the strength of producing uncorrelated features(solid lines)in the low frequency channels(ch-1) as shown in Fig. 4(d) and (e). Other channels'(ch-2 to ch-6) features, are highly overlapped like the surface density features, failing to preserve class-boundary as shown in Fig. 4(f). However, both descriptors(contrast and cortex) well classify the three classes in mos31 image as shown in Figs.3 and the corresponding histograms also reveal lesser correlation. table 2. Fig. 12 re ects the performance gain of our This observation supports the basic requirement of integration method. minimal correlation among various features. Feature integration using vector tagging overcomes class-contiguity problem and produces robust classi caTable 2: Average overall accuracy tion. Correlation co-eÆcient based features combination M ethods Avg. overall accuracy (%) is also attempted. Figs. 5-10 show the performance of 20 images 50 images our method of feature integration visually. Scene1 im10-Brodatz 20-Brodatz age of Fig.6, shows the contrast features fails to extract 10-VisTex 30-VisTex cloud(top-left corner), whereas combined features successfully identify these classes. Cortex 94.9 90.7 Fig.11 shows a visual comparison of the classi ed images Brightness 79.8 62.5 obtained from vector and correlation based techniques. Combined 97.7 93.4 Clearly vector integration appears better. 5.1.1
Performance Study
The performance of our fusion technique is analyzed using confusion matrix formation. Individual and overall class accuracy are computed from the classi ed images obtained from cortex, brightness and fused features. Table 1 shows only the overall classi cation accuracy of 20(10, Brodatz and 10, VisTex) sample images obtained from the individual and combined methods. The average overall accuracy on the selected samples(the same samples for training) are shown in
6 Conclusion
A novel approach of image texture classi cation has been proposed. The main observation is that all aspects of texture can not be captured by a particular category of techniques or features. Hence fusing di erent types of features is inevitible. Though in mosaic texture images, boundary appears unsmooth, our cortex transform based statistical features better capture
1468
Overall classification accuracy(%)
Graph for overall class-accuracy for 20 images Accuracy(Cortex) Accuracy(Brightness) Accuracy(Combined)
120 100 80 60 40 0
5
10
15
20
Image name indices
Figure 12: Graph showing the performance feature integration the textural features including microtextures. On the contrary, brightness based features like DSD, NSI, and NFI produce smoother boundary with computational adavantage but they fail to aggregate pixels in some natural scene images. Fusing features of di erent categories often integrates the goodness of individuals to produce robust classi cation like various images(mos31, mos41, scene1, scene2, ower1 etc). However, vector fusion appears better than correlation based fusion in our preliminary studies. More investigation is necessary, to conclude the robustness of the correlation based fusion technique. So the future subject is to conduct more analysis on the feature correlation and integration with application to multi-sensor images. We also investigate the performance of our method under various experimental conditions.
Acknowledgements
This work was supported by The Hori Information Science Promotion Foundation, Japan. We would like to thank to Dr. Kudo, Dr. Matsumoto, and Dr. Takeuchi, Dept. of Information Engineering, Nagoya university, for their valuable technical advices.
References
[1] Solberg A. H. S.,"Texture fusion and classi cation based on exible discriminany analysis. In Proceedings of 13th IEEE International Conference on Pattern Recognition, vol.2, pp.596-600, 1996. [2] Solberg A. H. S. and Jain A. K., \ Texture fusion and feature selection applied to SAR imagery", IEEE Transactions on Geoscience and Remote Sensing, vol.35, no.2, March 1997. [3] Huang Y., and Chan K. L., \Multi-model Feaure Integration for Texture Analysis", In Proceedings 1469
of the 5th Asian Conference on Computer Vision (ACCV), Melbourne, Australia, vol.2, pp.556561,Jan.22-25, 2002. [4] Jain A. K. and Farrokhnia F., " Unsupervised Texture segmentation using Gabor Filters", Pattern Recognition, vol.24, no.12, pp.1167-1186, May 1991. [5] Goresnic C., and Rotman S. R., "Texture Classi cation Using the Cortex Transform", CVGIP:Graphical Models and Image Processing, vol.54, no.4, pp.329-339, July 1992. [6] Dunn D. and Higgins W. E., "Optimal Gabor Filters for texture segmentation", IEEE Trans. Image Processing, vol.4, no.7, pp.947-963, July 1995. [7] Zhu, C., and Yang X., \Study of Remote Sensing Image Texture Analysis and Classi cation using wavelet", International Journal of Remote Sensing, vol.13, pp.3167-3187, 1998. [8] Lu C. S., Chung P. C. and Chen C. F., "Unsupervised Texture segmentation via Wavelet Transform", Pattern Recognition, vol.30, no.5, pp.729-742, 1997. [9] Scheunders P., "A multi-valued image wavelet representation based on multi-scale fundamental forms", IEEE Trans. on Image Processing, to appear 2001. [10] Pentland A. P., \Fractal based description of natural scenes,IEEE Trans. Pattern Analysis Mach. Intell., PAMI-9, pp.661-674, Nov.,1984. [11] Chen C.C., Daponte J. S., and Fox M. D., "Fractal feature Analysis and Classi cation in Medical Imaging", IEEE Transaction on Medical Imaging, vol.8, pp.133-142, 1989. [12] Super, B.J. and Bovik A. C., \Localized measurement of Image Fractal Using Gabor Filters," Journal of Visual Communication and Image Representation, vol.2, pp.114-128, 1991. [13] Bashar M. K., Mohan B. K., Shevgaonkar R. K., Ohnishi N., "Unsupervised Texture classi cation of Images using Cortex Filters", The proceedings of the fth Asian Conference on Computer Vision (ACCV), Melbourne, Australia, pp.828-833, Jan.22-25, 2002. [14] Deer P. J., and Eklund P. W., "On the Fusion of Image Features", NEC Research Index found in http://citeseer.nj.nec.com/162546.html.
