May 5, 1992 - vectors in the new space are considered as the feature vectors to be input to a neural network for classification. The pro- perty of the mapping ...
Conclusion: Although this process is inherently more complicated than standard e-beam lithography techniques, we believe our process is quite applicable to those universities and industrial laboratories who d o not have access to expensive e-beam systems. We present this technique to those researchers in the microwave or optical communities who would like to be able to fabricate structures on a nanometre scale, but are otherwise resource limited.
where d i j = xIrjJ, i, j = 1, _ .., q. and 11 11 denotes the 2-norm. Let D denote the matrix whose ijth element is drj.(x', . . . ,x') = (y,', . . . , y')DT is given. T denotes the transpose of the matrix. D is nonsingular for the linear independence of x', . . . , x4 and y', . . . , y'. Let e', . . . , e4 denote an orthonormal basis in R'. We construct a q x n matrix P , which is a mapping from R" to R4,by e', . .. , e4 and y ' , . . . , y y in the way of outer product
Acknowledgments: This work was sponsored by the Communications Satellite Corporation. The authors would like to thank Dan Wilcox for his SEM assistance. 5th May 1992
,=1
A set of vectors U', . . . ,'U in R' is defined by the linear combinations of e', . . . , e' in terms of d,, i,; = 1, . . . , q :
G . M. Metze, P. E. Laux and S. Tadayon (COMSAT Laboratories,
Clarksburg, MD 20871, USA)
4
U' =
1d L j d
i = l , ...,q
(3)
,= 1
The mapping P has the following properties: (i) Lemma I : Px'
=
xi, i
=
1, . . . , q.
B=l
(iii) Lemma 2 : P r P x = x for any x P P T u = U for any U E R4.
Indexing term: Neural networks A neural network approach to classification using features extracted by a mapping is presented. When the number of
sample dimensions is much larger than the number of classes and no deviations are given but the means of classes, a mapping from class space to a new space whose dimensions are exactly equal to the number of classes is proposed. The vectors in the new space are considered as the feature vectors to be input to a neural network for classification. The property of the mapping that the separability of the original classification problem does not change is described. Simulation results for object recognition are presented. Introduction: The first step in a neural network approach to classification is feature extraction [l-31. The class space is generally partitioned into q regions with the centres which are training samples representing the means of the q classes. If no deviations of the classes are given, the Euclidean distance can be used as the distance measure between an unknown sample and one of the centres in a nearest neighbour classifier. In other words when an unknown sample x is to be classified, the nearest neighbour of x is found among the pool of all q available training samples and its label assigned to x. When the number of dimensions of the class space is much larger than the number of classes, a mapping used for the feature extraction for classification is proposed, which is the subject of this Letter. When an unknown sample x is to be classified, x is projected onto the subspace spanned by the training samples and converted to a new vector whose dimensions are exactly equal to the number of classes. The new q-dimensional vector is considered as the feature vector to be presented to neural network to determine which class sample x is assigned to. It was proved that the mapping does not change the separability of original classification problem in the sense of nearest neighbour classification. Mappiny: Suppose a set of trainl'g samples X I , ..., x y E A c R", q < n, which represent the centres (means) of class 1 to class q, respectively, are linearly independent. A is an n-dimensional bounded subset. Let y x ' , . . . ,x') denote the q-dimensional subspace spanned by xi, .. ., x'. An orthonorx') is obtained by the Gram-Schmidt mal basis of U x ' , . . _, method: =
and PTu'
C=,
Yi S u n
Y'
= U'
(ii) Proof of lemma I: Px' = c,'= ByJTxi= c,'= 1 d '-I eJ = ui. ' PTu' = y'dT dike' = d i t y k= x i .
NEURAL NETWORK APPROACH TO CLASSIFICATION U S I N G FEATURES EXTRACTED BY M A P P I N G
xl/llxlll
y L = ( x i - 1x- 1d f J 9 ) / l l x i - , x- Id i j $ l l J=
I
J=
ELECTRONICS LETTERS
-
i = 2 , ...,q
18th June 1992
_-
(1)
1
Vol. 2 8
No. 13
E
L(xi, .. ., x"); and
(iv) Proof of lemma 2 : Any x in y x ' . . . ., x') can be expressed by the combination of .y', . . ., .p in the form of x = y,y=, -. . p l.y ' . Hence, P T P x = y i d T c,'= dy" Dl y' = y'eiT Die' = p l y ' = x . In the same way, we can prove PP'u = U for any U E R'.
E=
E=
cf=l
E=
(v) Property I : For any xi there exists one and only one U' = Px' E R4,i = 1, . . . , q, and vice versa. (vi) Proof of property I : Existence is proved by lemma 1 and the uniqueness is guaranteed by the fact that D is nonsingular. (vii) Property 2 : The Euclidean distance between U' and 'U is equal to the Euclidean distance between x' and X I , i.e. l(ui - uj/l = llxi - xjll, i,; = I, ..., q. (viii) Proof of property 2 : The proof is easily obtained after examining lemmas 1 and 2. (ix) Properfy 3: For any x E R" and any x i , if IIx - xi 11 Illx - xJll, then /lu - u'(I 5 Ilu ~ ' 1 1 , where U = P x , I, . . . , q . ~
;=
(x) Proof of property 3 : Any x in R" can be orthogonally decomposed as x = x p xq, where x p E L(xi, . . .,x'), and xq E LT(x', _..,x'). LT(x', ..., x') is the orthogonal complement of ,!,(xi, . _ . ,x'). xq is orthogonal to x p , therefore we have (Ix - x'Il2 = /lxp- xiIl2 + I / x J 2 and llx - xjl12 = l/xp- xJ1I2 + I1x 112. Because IIx - x'I/ IIlx - x'll, then (Ixp- x'll I/Ixp - x'l. Notice that P x = P x , and from lemmas 1 and 2 we obtain IIu - uijl = /lxp- xil/ and IIu - u'll = I/xp- xJll; thus I/u - dl1 for j = 1, . . . , q. we can conclude that 1\11- 111' 1 I These properties show that the training samples X I , . . . , x4 can be uniquely mapped to U', . . . ,'11 by P and the Euclidean distance between any two of them remains unchanged. If x in R" is assigned to the ith class by the nearest neighbour classifier its corresponding U in R' will also be assigned to the ith class by the same classifier. Meanwhile, we must emphasise that the dimensions of U are much less than the dimensions of x . U can be considered as a feature vector of x to be used to classify x . Obviously, the parallel structure of the mapping is suitable for constructing classifiers combined with neural networks.
+
B P neural network classifier: Fig. 1 shows a proposed neural network based classifier consisting of the previously described mapping followed by a BP neural network. The output function of a neuron in the net is a sigmoid function described by
1263
f(a)= l/(l + e-'). T o avoid the truncating effect of the nonlinear function it is necessary to linearly transform U', . .. ,'U to a set of vectors belonging t o [ - 1, 114.We define
w = gP
(4)
and h
=
[(M
+ N ) / ( M - N ) , .. ., ( M + N ) / ( M - A')]'
(5)
where M = {U;}, N = min,,,,jrs { U { } and g = 2/ ( M - N ) . If x E A c R", the input vector of the first layer is gPx - h, which is approximately in the range [ - 1, 11'. It can be shown by simple investigation that the operations of g and h do not affect classification results. Both the input and the output layers contain q nodes. There is no precise way to choose the number of hidden layers and the number of nodes in every hidden layer. In this Letter, the BP network has one hidden layer and q nodes in the hidden layer. The interconnection matrices B and C between the input and hidden layers and between the hidden and output layers, respectively, are adjusted according to the back-propagation algorithm [4]. For training, all outputs are set to zero except the desired output which is set to one. The training feature vectors cyclically presented to the first layer are gu' - h, i = 1, . .. , q. The co-ordinate basis of Rq is selected as e', . . . , e' to construct P. XI
500 noisy images are tested. Every 100 noisy images are generated from one of the noiseless images. The percentages of correct classification are 100~0%, 100.0%, 63.0%, 55.0%, 35.4% for 32, 16, 8, 4 and 2dB SNR images, respectively. The simulation was carried out on an AST-386 microcomputer.
Conclusion: A neural network approach to classification using a mapping to extract features has been presented. The method is available for the case in which the number of sample dimensions is much larger than the number of classes and no deviations but the centres of classes are given. 5th May 1992 Yi Sun (Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, Shanghai 20030, People's Republic of China)
References L., SAYEH, M. R., and TAMMANA, R . : 'A neural network approach to robust shape classification', Pattern Recognition, 1990,
GUPTA,
23, (6),pp. 563-568 KHOTANZAD, A., and LU, I.: 'Classification of invariant image representations using a neural network', I E E E Trans., 1990, ASP-38, pp. 38-44 GRACE, A. E., and SPA", M.: 'A comparison between FourierMellin descriptors and moment based features for invariant object recognition using neural networks', Pattern Recognition Lett., 1991.12, pp. 635-643 RUMELHART, 0. E., HINTON, G. E., and WILLIAMS, R. 1.: 'Learnlng internal representations by error propagation'. In 'Parallel distributed processing: explorations in the microstructure of cognition, Vol. l'(M1T Press, Cambridge, MA, 1986)
COMMENT Fig. 1 Proposed structure of BP neural network classifier Simulation results: The images used are 64 x 64 binary images. Fig. 2 shows five objects used to train the BP neural network classifier. They represent the centres of the five classes. In this case n = 4096 and q = 5. 15 nodes are used. In
MINIMAL REALISATION FOR SINGLE RESISTOR CONTROLLED SINUSOIDAL OSCILLATOR USING SINGLE CCll
M. T. Abuelma'atti
1920121 Fig. 2 Five objects
addition to the noiseless image set, five other sets of noisy images with respective signal to noise ratios of 32, 16, 8, 4 and 2dB were constructed from the noiseless images. The noise was added by randomly selecting some of the pixels of a noiseless image and inverting the values. The random pixel selection is performed according to a uniform probability distribution. The signal to noise ratio is computed using 20 log [ ( n - L)/L]
This Comment relates to a recently published Letter [l] which presents a single CCII single resistor controlled sinusoidal oscillator. This oscillator has already been published in the literature. Moreover, another oscillator circuit which has similar properties is presented. Celma et al. [I] present a truly canonic second-order RC sinusoidal oscillator using a single second generation current conveyor. The proposed structure uses two capacitors and only three resistors. The condition and the frequency of oscillation can be adjusted independently using two control resistors. This oscillator circuit, shown in Fig. l a , has already been published in the literature (Reference 2, case V, Fig. 6, and
where L is the number of pixels which are different between the noisy images and the noiseless images. Fig. 3 shows one image with different signal to noise ratios. At each noise level
.
1
.L
I
15481110 Fig. 3 One image with different noise levels From left to right: SNR IS 32, 16, 8, 4 and 2dB
1264
b
Fig. 1 Single element controlled sinusoidal oscillators a, h . see text ELECTRONICS LETTERS
18th June 1992 Vol. 28 No. 13
