dynamic competitive probabilistic principal

International Journal of Neural Systems ©World Scientific Publishing Company

DYNAMIC COMPETITIVE PROBABILISTIC PRINCIPAL COMPONENTS ANALYSIS EZEQUIEL LÓPEZ-RUBIO, JUAN MIGUEL ORTIZ-DE-LAZCANO-LOBATO

School of Computer Engineering. University of Málaga. Bulevar Louis Pasteur, 35. 29071 Málaga. Spain Phone: (+34) 95 213 71 55 Fax: (+34) 95 213 13 97 {ezeqlr,jmortiz}@lcc.uma.es

We present a new neural model which extends the classical competitive learning (CL) by performing a Probabilistic Principal Components Analysis (PPCA) at each neuron. The model also has the ability to learn the number of basis vectors required to represent the principal directions of each cluster, so it overcomes a drawback of most local PCA models, where the dimensionality of a cluster must be fixed a priori. Experimental results are presented to show the performance of the network with multispectral image data. Keywords: Unsupervised learning, local PCA, probabilistic PCA, competitive learning

1. Introduction Vector quantization (VQ) is a method for approximating a multidimensional signal space with a finite number of representative vectors (code vectors), which form a codebook. The aim of competitive neural networks is to cluster or categorize the input data, and they can be used for data coding and compression through vector quantization. This is achieved by adapting the weight vector of each neuron in order to estimate the mean or centroid of a data cluster. Hence, vector quantization is a useful tool in the field of cluster analysis [16]. Competitive learning is an appropriate algorithm for VQ of unlabelled data. On each computing step, only the winning neuron is updated, i.e., the neuron with the weight vector which is closest to the current input vector. Ahalt, Krishnamurthy and Chen [1] discussed the application of competitive learning neural networks to VQ and developed a new training algorithm for designing VQ

code-books which yields near-optimal results and can be used to develop adaptive vector quantizers. Yair, Zeger and Gersho [24] have proposed a deterministic VQ design algorithm, called the soft competition scheme, which updates all the code vectors simultaneously with a step size that is proportional to its probability of winning. Pal, Bezdek and Tsao [13] proposed a generalization of learning vector quantization for clustering which avoids the necessity of defining an update neighbourhood scheme and the final centroids do not seem sensitive to initialization. Xu, Krzyzak and Oja [23] developed a new algorithm called rival penalized competitive learning which for each input not only the winner unit is modified to adapt itself to the input, but also its rival delearns with a smaller learning rate. This approach has been further exploited recently in [3]. Ueda and Nakano [20] presented a new competitive learning algorithm with a selection mechanism based on the equidistortion principle for designing optimal vector quantizers. The selection mechanism enables the system to escape from local

2 Ezequiel López-Rubio and Juan Miguel Ortiz-de-Lazcano-Lobato

minima. Uchiyama and Arbib [19] showed the relationship between clustering and vector quantization and presented a competitive learning algorithm which generates units where the density of input vectors is high and showed its efficiency as a tool for clustering color space in color image segmentation based on the least sum of squares criterion. Finally, competitive learning has been recast in a probabilistic framework [4]. This is achieved by considering that the samples are assigned a class by a random labelling process. Then a competitive learning method is shown to have a low classification error probability. However, data analysis with competitive networks has severe limitations, because the model only provides information about the mean of each cluster. The Principal Components Analysis is a multispectral data analysis technique, which is aimed to obtain the principal directions of the data, i.e., the maximum variance directions ([7]; [9]). Hence, if we have a Ddimensional input space, the PCA computes the K principal directions, where K