«Potts-perceptron» with. N. Q-states input neurons and one. Q' states output neuron, we compute the maximal storage capacity for unbiased pattems. In the.
Phys.
J.
France
I
(1991)
1
l109-l121
AOOT
1991,
PAGE
l109
Classification
Physics
Abstracts
05.20
87.30
Storage
capacity
Jean-Pierre
(')
Lab.
Cedex
Nadal
f) Dept.
(')
Physique
de
05,
of
Potts-perceptron
a
and
AJbrecht
Statistique (*),
f)
Rau
Ecole
Supdrieure,
Normale
24
rue
Lhomond,
F-75231
Paris
France
Theoretical
of
Physics,
University
of
Oxford,
I
Keble
Rd.,
GB-OxfordoXl
3NP,
G-B-
(Received
22
January
1991,
accepted
in
final form
30
April 1991)
networks where each be in consider the properties of « Potts » neural Abstract. We neuron can with different For «Potts-perceptron» N Q-states input and Q states. a neurons one Q' states output pattems. In the storage capacity for unbiased compute the maximal neuron, we be stored is found to be proportional to of pattems that large N limit the maximal number can f(Q'), where f(Q') is of order 1. N(Q I
Pom-perceptrons,
1.
The
Hopfield
and
was
linear
machines
and
winner
take
all
systems.
[I] Of a formal neural network used having two possible states, neurons extension of this model of analogy with spin glass systems. A natural associative taking more than instead of having is to consider two states : memory neurons neuronal described by an Ising like variable, one has a Potts [2] like variable. One then state obtains a neural network Potts [3], which is to the Potts Glass [4] what the Hopfield model is to the Spin Glass model. The physics of Potts neural networks has been studied statistical for networks with Hebbian leaming rules ([3, 5-7]). In these studies, one is neural attractor considering a Potts-Attractor Neural Network ~PANN) : each whose activity can neuron, take Q different values, is connected to every other and the stimulus (a to a neuron, response given initial configuration of activities) is the which the dynamics of the natural attractor to leads. Clearly one Hebbian learning rules and in particular also consider net can non non symmetric couplingsfor the of binary One also consider Pottsas case neurons. can feedforward networks. Perceptrons (PP), that is ~possibly multilayer) In that the case, of possible layer. the following number In will consider only states may dirtier from layer to we the simplest that is one input layer with Q-states and output layer with case, neurons one Q' states (and no hidden layer). Such systems are well known in data analysis neurons literature under the of «linear machines» [8]. The binary [9] is one perceptron name particular case. In fact the perceptron algorithm, as well as its variants, is easily adapted to multistates ([8, 10]). neurons (*)
based
Laboratoire
model on
an
associd
au
CNRS
(URA1306)
et
aux
Ulfiversitds
Paris
VI
et
PaHs
VII.
II10
JOURNAL
PHYSIQUE
DE
I
lNf 8
considering such systems are manifolds. Consider the of a I) case just mentioned~ such allows deal with multiclass system to classification tasks, and with strings of data made of N items~ each one taking Q possible different values. Here few examples. In image processing, Q Q' would be the number are of grey each input node would be in one of the levels. In the analj,sis of DNA sequences~ the might be binary for codon Q 4 letters~ A, T~ G or C and output exxon see versus interested in predicting the secondary structure e.g. [I I]). In the analysis of proteins, if one is number from the of amino acids, Q would be 20, the of different acids, and sequence amino (a helix, p sheet or randomj iii Q' would be 3, the number of different possible structures Consider feedforward with N inputs and Q'-state output One network one neuron. a can binary winner take updating consider Q' with all rule, the output as neurons, a neurons » receiving the highest input field will fire. Such systems commonly and only the neuron are both for biological modeling and engineering applications. There are found in the literature, the competition models of self-organisation based between in particular (see e-gon neurons mentioned the fact that of [l2]). iii) We have already such natural extension system are a addition, In for a from the physicist point of view. multidass binary neural networks classification task (including the two class, binary casej, the use of a Potts perceptron neuron theory (see e-g [8] (as precisely defined in the next section) can be justified in Bayes decision indications that p.16 and [11]). iv) Lastly we note that, on the biological side, there are coherently, having a small of possible cortical columns behave number coherent [13]. states neural network insight on the behavior of a network of Hence a Potts-attractor may give some The
for
motivations
feedforward
As
network.
=
=
cortical
columns.
Here
will be
we
classification the neural the
pattems~ present the
v'
network.
define
first
us
specifying
encoding
compute
the
chosen
be
for
neurons)
associative
algorithm. the
maximal
fractional
limit
biased
to
properties. the
volume
will
We
patterns. extended
patterns.
will
We
show
capacity
storage
jPANNj or particular we will storage capacity of a of the weights which study to unbiased our memory
In-
its
results.
In
section
relevance The
?
we
to
the
details
of
Appendix.
the
dynamics.
and
model
the
inputs (Q-state is given bv
[14] for
will
in
given
for
Gardner we
the
invariance gauge section 3 we present
In
be
networks
leaming computing
randomly computation can
discuss
will
neural
Potts
of
set
and
invariances,
Gauge
Let N
by Namely,
learning of a although the model
with without
initiated
network.
dynamics of the computation the
2.
(PP),
approach
follow Potts
realize
concerned
tasks
simplest
the
and
one
h~
=
feedforward
Q'-state
output
jj (js',s)
3~~
the
case,
The
neuron.
Potts
local
perceptron field
at
with state
(lj '
J.~
The state
synaptic s~
on
ii..Q)
.v~
jn~,j
I... =
lip,
T
is
((s', s)
matrix the
and
N),
s'~
with
defined
the weight of a signal coming from node j, which is in processing unit. otherwise indicated ii.. N ), Unless j ~ A input activities denoted (I.. Q'). of will be pattern probabilist il. The decision rule Q ). at «temperature» ~ a
indicates
the
of
s'
state
n~
by
=
s[~~
s' =
with
probability
exp
jj ~
p (h~,
exp p
(fi,.
H~,
0, j
(2j
M
STORAGE
8
the
where
by
thresholds.
the
are
OF
CAPACITY
s[~~
At
(s(
=
POTTS-PERCEPTRON
A
rule
is
s()
0~, Vs' #
@~,) h~,
h~~
decision
the
temperature,
zero
II II
(3)
o
winner-takes-all consider This is a « rule : the Q' state output neuron » can as one s' receives the input field h~, defined by (I), and only the Q' binary neuron neurons neuron with the highest local field will become active. The generalization to N' output neurons or to a is straightforward. Indeed, fully connected net (PANIQ~ for which N' N and Q Q' for the basic properties considered in this section as well for of binary as for the case neurons, the analysis in the next section, one can always focus on one particular (output) storage which is equivalent to consider with only one perceptron output a neuron. neuron, invariant under the following translations Updating rules (2) and (3) are =
( (S', S )
( (S', S )
-
@~,
and
=
0~,
-
(S) j
(4)
uo
+
under
((S', S) @y
where
the
uo,
local
each
to
output
uj(s) field
thresholds.
In
of
set
addition
translations
updating
the
positive
gauge
invariances
possible
One
s(,
strictly
and
authors
to
set
to
([8, iii). another
make
choice zero
It is
choice
real
number
allows
namely
under
(4) add
transformation
first
absorbed
be
can
independent
is
the
in the
(6)
redefinition
global rescaling
a
of the
fields by
local
a
of
:
j3-p/A
(7)
parameters
by fixing the gauge ». particular output state
@~,-A@~,,
A.
reduce
to
that
however
input
state
so
( (s', so).
and
choice
gauge
any
of
number
the
is to pick one particular all the couplings ( (s(, s) clear
The
input pattem, couplings (5) modifies invariant
is
rule
(6)
numbers.
then
j(s',s)-AJj(s',s), any These
(5)
of the
the
on
which
pattern,
vj (S')
+
£ uj(s')
@y
-
arbitrary real are although a function
input
of the
((S', S)
-
vj(s')
which,
term
second
independent
the
the
and
a
The
states.
term
for
+ U
and
one
This
is
will
do.
In
a
choice the
made
by
several
section
next
we
will
:
Z((s',
s
o ;
vs'
(8)
=
o ;
vs
=
(9)
)
s
Z((s',
s)
~.
by considering the dynamics, is more natural. we argue as now dynamics of the network, whether it is an neural attractor net or a quantities. This can be easily shown should only depend on gauge invariant dynamics. For simplicity, consider a perceptron with NQ-states inputs and (Q'= 2). In that case one can write the updating rule (3) as
which,
feedforward
The
"
with
w~(s)
=
((2, s)
((I, s)
"
(h) h
sgn
and @o
=
@~
=
£
j,
w~(n~) and
«
for one
the
binary
±
I is the
step
output
(10)
@o
=
net,
one
binary
output.
Suppose
JOURNAL
II12
that
now
for
As
the
binary
the
variable~
presented
perceptron here
that
so
input
the
is
pattern
n~
=
n~
=
[15] one probability
version
I
of
M
teamed
a
pattern,
8
pattern
say
probability f
f'
getting
for
(l j
Q
that~ for large N,
show
can
~
probability
with
# n~'
s
noisy
a
with
n~'
PHYSIQUE
DE
the
~'"lfij
input
the
field is
Gaussian
a
is
output
correct
~~~~
where m
H(x)
i13)
Dy,
=
~
Gaussian
the
being
measure
by
indicated
Dy The
first
two
conditions
with
the
(h)
of h,
cumulants
(h~)~ (the
and
fi,
bias
same
(- y~/2 if,.'2
exp
=
j14)
ar
being
averages
I li(nj')l~
tn)
(i
+
=
j
tn)
the
renormalized
couplings
initial
and
I li(.I)l~ ,
,
and
possible
jj ((n~')-0
=m
J
tn(i
all
given by
are
(h)
lh~l~
over
threshold
j(s)ww~(s)
~
by
given
are
list
zw~(i)
Q
,
0
The
parameter
noisy
the
which
Hi
appears
w
characterize
above
the
=
~
is
the
natural
in
the
and
pattern
the
(17)
~
value
~~
pattern
between
II, ,,
m
It
correlation
m(Q3~i
l ;
takes
the
(16)
version
(Q3~ and
jjw.,(t). Q~,
Ho
case
generalization of binary
of neurons
the
usual
IQ
2 =
~
(18)
Q
=
«
).
overlap It
takes
between the
value
the
if
input there
is
string no
with the pattern. It is important to notice input, and 0 if the input is uncorrelated concentrated dependance in the initial conditions is in this The above parameter. easily be generalized finite also Note that, the of temperature. to can in case
and
noise
in
that
all
formula an
the
the
the
j12)
attractor
M 8
CAPACITY
STORAGE
characterizes network, (12) fully [15]. conclude, it is thus natural more
neural
dynamics
the
II13
POTTS-PERCEPTRON
A
OF
the
in
of
case
highly
a
diluted
network To
hy
and
explicitly
to
fixe is
=
threshold»
particular,
point
this data
way
=
a
it is
useful
to
make
in
the
the
as
for
field
neural
a
comments
some
input layer.
string of Q nodes. The direct interpretation Q-bit string with only the nj-th bit on. In site j.
at
This is the
consider
most
of
choice
common
representations. Q nodes
other
configuration
Q
the
the
((s') j, Q
the
(xf,n I. representation,
vectors
thermometric
«
(I)
of
»
whose
vector
different
for
values
3.
forming
where
a
given
are
of the binary representing the
case
capacity
choice
of
and
of
a
the
letters
first
is
is
field
coupling
the
to
=
RQ. A
all
What
on
the
can
be
from But
as a
node
one
by
can
given
a
have
couplings layer of any
the
choice,
that
definition
0
from
and
shown
is
(19)
there
neural
adopt for
s
that
in the
the
local
the note
we
that
thresholds.
of the
feedforward we
If
is
node
is
matters
just
example
otherwise.
coming
what
we
typical
and
nj,
coupling
(3) with the
rule
of I
s
The
».
that
each j, the activity is encoded the s-th state is represented by
((s',s)
base
a
for
a
for
encoding problem, (I) of the local
Q),
the
given j. reinterpretation of s'
that
case
stated,
Otherwise
considerations
Storage
I
is
component
s
((s').x".
equivalent, via a apply as well which the input data are strings of corresponds to take the updating choices (8) and (9). gauge
choices
the
in
encoding found in the literature. defined general case, is state n
xi
=
for
is
that
the
In
the
entries
of
Q )
=
((s',n)=
then
these
choice, standard
definition
xn=(g,s=i. with
(19)
l)
on
The
with NQ
net
a
also
as
=
represented
are
understood
s
fields
s) (Q3~,~~
by (8) and (9). This 2, corresponds to the
gauges
for
activity by
At
£ ((s',
=
local
the
Q' Q With these choices, the case « without spin like variable, ) ± I. the between the Q is the only situation which states : in symmetry preserves the optimal thresholds will unbiased be when leaming random, patterns, zero. that
perceptron, neuron
the
define
to
site
at
are
possible
all
that
Note
network
for
following, fieldi, and
thus the
Pom-perceptron.
consider with N inputs, each We now the storage capacity of a Potts having perceptron one will number Q possible states, and one Q' state output We the maximal of compute neuron. input-output pairs (n~P, j which random for there exists I N ), n'~ ), p I set p, a of couplings such that, for every p (p n'~ if the input is I,... p ), (3) is true with s[w =
=
=
=
number
pattern which
is
p.
equivalent
All
Potts
states
in
occur
the
the
thresholds The
pattern
average to
translational
with
pattems
are
the
probability,
same
is
indicated
by (... )
=0. f
J
Here
pattems
to
lQ8~~»-1)
the
of
set
the
Since ~.
unbiased,
we
can
set
all
zero.
freedom
of the
£ ((s',
field
local
s)
0 =
is
fixed
Vs' =
I
by Q'
(20)
JOURNAL
l14
Z((s',
PHYSIQUE
DE
s)
vs
o
=
M
I
Q'
i
8
(21)
=
~.
should
It
that
noted
be
the
constraint
the
scalar
of (20) for s' Q' degree of freedom
is
included
(21). We
in
=
only the .v' by
when
case
((s', s))~
£
is
consider
will
independently
fixed
for
(22)
Ny
=
each
j,s
The
constant
of
is
y
arbitrary,
course
y
the
From
each
For
invariance
gauge
there
pattem
expect
that
N(Q
i
the
we
maximal
) (Q'
see
i
),
(Q
i
that
there
that
will
this
that
see
is
the
=
a
which
(Q
be
to
can
I
) free
satisfied. will
stored
be
parameters.
Thus be
one
of
can
order
(24)
N
slowly
a
IQ'
I
need
that
I
a
(23J
only N(Q
are
take
we
)/Q'
i
patterns
with
case~
convenience
(Q'-
of
p~~~ iS
p~~~ We
later
inequalities
I
number
)/(Q'
i
for
=
(Q'-
are
but
varying,
bounded,
function
of
Q' only. the
s,;rem
entropy
method,
&[h~(n'" )
h
maximum
and
we
define
the
partition
function
as
dp (J)
Z =
h"(s')
Here
[14]
Gardner's
follow
We of
is
local
the
fl
fl
»
~j~~.~j
field
is
jj ((s',
h"(s') =
dp (J) is the
input
the
when
"
,~)
is')
(25)
x
pattern
p~
s) (Q3~_~~
),
l
in the space of interactions which is compatible with (?0)measure asking for minimal stability the of the (In Q Q' we are a 2~ case x. / defined differs in (25) by from stability factor the standard parameter as a x introduced by Gardner [14]). In order to evaluate the quenched average the distribution over of (ln Z) the replica trick. of the Assuming the validity of a entropy S patterns we use and shrinking the replica symmetric solution volume of interactions find for the to zero we capacity a (as defined by (24)) maximum storage and
(22). As stability
in
=
~~
_, "
H~ 2
Dy is the
Here
natural
[14],
~
(l
~° ~~~
~~~~~~ ~
Ho)Q'-
+
(Ho)~
Q'Ho
(14) and
measure
H, (_i')
«
~
Dt(t
m x
we
+ j'
have
+
x
introduced
I'
;
I
=
j,'2 ~~~~
(Ho)~
Q'(Q'-
Gaussian
Q'(Q'-
=
functions
the
(27)
0, 1, 2
i
derivation of (26) is given in the Appendix. The justification for restricting to symmetric calculation is the for the usual, Q Indeed, any Q' 2, case. same as continuously transformed solution other solution, be from be into the can any as can seen geometrical picture (one particular solution is a set of Q' vectors with well defined angles them, and one go through any between solution by global rotations). Details
the
of
the
replica
=
=
M 8
STORAGE
For
(x
a
=
Q'= 2 0) 2.
K
=
finds
one
one
=
=
limit
~
,
~
~
the
particular with in capacity analytically, and
Gardner,
storage
-1
$
~
II15
~'~~~'
~
~~~~
gets
one
«
~~
by
found
evaluate
can
/
large Q'
the
3
:
~
In
Q'
0 and
POTTS-PERCEPTRON
A
capacity
critical
the
recovers
one
For
OF
CAPACITY
(Q,
»
1)
DyiH~
=
(H~)2/Hoi
»
(29)
3.850.
a 3
Fig.
Maximal
I.
The
results
table I. In Table
capacity
for
4
2
-0
x
0 =
Q' as
function
a
Maximal
storage
capacity
at
K
that
critical of
about
capacity 3.85.
as
measured
However,
the
by
a
2
2.000
3
2.320
4
2.546
5
2.714
6
2.844
is
for a
K
0 =
function
shown
are
of
K
in
figure
I
and
Q'= 2, 3 and 4.
for
a(Q')
j.850
co
The
Q"
0.
=
o'
value
of
for solving the integral in (26) numerically figure 2 is shown the maximal capacity as
I.
lo
8
6 ,
thus
information
an
increasing function I(Q'), in nat
content
of per
saturating at (free) parameters,
Q',
a
is
I(Q') =
«
(Q')
in
Q'/(Q'-1),
(30)
JOURNAL
II16
PHYSIQUE
DE
lNf 8
I
25
j ~
>
~~
io
0
~
io
os
is
lo
is K
Fig.
Maximal
2.
capacity
as
a
function
stability
the
of
parameter
K
Q'
for
2.
3
and
4
infinity. Hence we find that~ by Kanter in the case of to one mentioned in the introduction~ there as are find of whenever which allow couplings there exists at algorithms type set perceptron to a solution ([8, 10]). such algorithms allow to respect any particular Moreover, least gauge one choice. To see this~ let us give the algorithm for the choice of the gauges (20) and perceptron with couplings. Then, the following is repeated until (21). One starts convergence zero Pick a pattern p at random. For any s'# ii'" such that hf ~ hf~, make a learning step for every j and every s by
decreasing function of Q', going to zero a qualitatively, the optimal behavior with Q' is Hebb rule. It is important that, to note a
/(n'", s)
((s', It is
clear
that
at
each
s
j
time
step
the
storage
-
-
the
similar
i(n'", s) j(s',
Q'
when
is
to
goes
found
the
(Q3
+
i
nj
(Q3s,n/
s
couplings
current
1) will
(3'
satisfy (20)
and
(?1).
Conclusion.
In the
this
article
we
maximal
analysed
number
of
pattern
that
capacity can
be
of
a
Potts
We
perceptron.
proportional function of Q'. Our
stored
is
to
N
Q
have
obtained
I ), the
being a slowly increasing and bounded calculation can generally, generalized to other cases~ e-g- the case of biased More it is clear patterns. of the analysis done for the binary Q' could be generalized to 2 perceptron (Q most it interesting consider function than Potts In particular would be other to cost perceptron.
a
=
number
of
errors,
as
considered
that
prefactor easily be that the
=
the
consider the above saturation. here, and to properties (We method, simpler than the replica techniques, has been derived
recently ([19]) a function. in order to study binary perceptrons at and above criticality, for any choice of cost checked that it can be easily applied to Potts-perceptrons.) In practical applications We have of neural networks, take all systems currently used. This is presently the main winnner are of for pursuing the analytical study such systems. reason note
that
very
M 8
STORAGE
OF
CAPACITY
POTTS-PERCEPTRON
A
II17
Acknowledgements. Normale Supkrieure for their kind of us (A.R.) thank the group at the Ecole wants to further acknowledges financial by the hospitality during his stay there. He support Studienstftung ties dbutschen Volkes, the SERC and Corpus Christi College, Oxford. This and BRAIN twinning (ST2J.0312.C(EDB) work has been supported by EEC contracts discussions. ST2J.0422.C(EDB)). We would like to thank Marc M6zard for several
One
Appendix. order
In
(In Z)
evaluate
to
we
~
identity
the
use
(
till z)
=
I
n
=
and
measure
the
on
fl
m
d
dA(a mm
I
exP
x
js~
fi
la
Ala
(it)
N
«
introduced
we
A(~
dP (J~)
(I)
0 ~
,~
~,j,n,~~
p.a
a
d4a
fl
fl
dP (J~)
z"
The
I
consider
thus
where
/
=
~
n
We
)
Z"
lim
m
£ ( ((n'~, is
Q~fl
pass'
(f(s',
)
)) (Q3~, ~
s
l
)
(iii)
~
couplings
(f(s', s )
s
by
defined
'
£ ((s',
~jS'
s
)
S
fl
£ /f(s',
~jS
S'
)
s
x
jj /f(S',
§
~
~
S)~
~~'
~~
Ny
~~~~
~
where will n
V is
a
compute
limit
normalization
(Z"),,
the
constant
and
average
of
the
the
over
nN
the
patterns
are
unbiased
one
(Q has
f
I
normalization
distribution,
pattem
in
(22). the
We
small
=
l
~~
) (Q8
performing
the
average
exp
nN
'
the
over
(Q
I
) f =
#
~
~
Q3 f
fl fl ldq()~'exp (G~ a.b
v.
(vii)
l ~
gets
one
n~P
))
J
(vi)
0 f
l
~~
f
J
(vj ~
l
~~ J
((Q8~
jZ"j
:
(Q3~
Then
fixing the
constant
:
exp
Since
y is
Z"
+
Gj)
(viii)
JOURNAL
1118
PHYSIQUE
DE
M
I
8
with
G~
exp
=
fl ~
x
fl ~~~
~
«
dA
d,~(~
j~
exp
(- I,;[~ A j~)
I
-~(a
x
$fi
~
£
exP
x
ah,~,
x)b lqli~
qli"'~
qli~
~'~
+
qlfi"I
fix)
~~~,
and
exp
G
j
ill
=
dp (J~)
x
~
ZJ~(S' a
~~
~
','
~~
Ny
s=1+> fl ~-~
~
k
with
critical
~
Q'
'
+
gives
which
~~~
x
t
taken
are
fl
~~~
~'
(X,
s'=i~-K
jj i=i
~~~
(3' X~.j
we
k
x
~'~'~' ~~
~~
(x~,»j
(Z' X~ +
following
express>on
for
the
capacity
i1
i
Q'-1
~
Pmax
Qj'
Q'~ k
~_j
Q '~ ~~~ ->
~~~ x
~
j
i
~
j«
=i
Qj'
~~~,
~-~
-i~i
,
~
~~~
~
-~
2
~'
jj ~.~
jj z) k+
,'Q' ~
~~
,
~~
(~X'lll)
bt 8
STORAGE
Introducing
Gaussian
a
CAPACITY
variable
in
OF
order
~
~~
(
to
decouple
~~
(~~~~)
one
sum
can
k
over
kA~
using the
and
~~,,
identities
'
(I
KA
A)~~'
+
(xxix)
k
i
and
the
K =
k
l12I
POTTS-PERCEPTRON
A
finally
arrives
expression (26)
the
at
for
the
Lent.
55
critical
capacity.
storage
References
[Ii [2]
WU
[3] [4] [5]
[6]
J. J.,
HOPFIELD PoTTs
B,,
R.
F. Y.,
SHIM
D.,
DUPONT
G. M.,
KIM
R. O.
DUDA
MINSKY
1990
S.,
RUMELHART
PERETTO
GARDNER KRAUTH
KEPLER
and
P.
VAN
CHoi
and
D.
P. E.,
HART
and
S. I.,
LAPEDES
DAYHOFF
[16]
(1982)
54
(USA) 79 (1982) (1952) 106j
2554.
235.
H., Phys. the
Sitges
J.
Phys.
Rev.
Conference
(1985)
304.
Neural
on
networks
(Sitges,
J.,
MOURIK
M.
Y.,
1990
Seoul
Classification
Pattem
AM
(1991)
National
Univ.
and
Scene
1065.
preprint
(SNUTP 90/33). York Wiley, 1973)
Analysis (New
5.
1986) [13] [14] [15]
and
M.
GALLANT
BRUNAK
[12]
48
1990)
June
Chap. [10] [I Ii
Sci. Soc.
I., Phys. Rev. A 37 (1988) 2739. GRoss SOMPOLINSKY D. J., KANTER I. and CooK J., J. Phys. All (1989) 2057. BOLLt D. and DUPONT P., Proceedings of BOLLt
[9]
Phys.
Mod.
Rev.
Acad.
Phil.
Camb.
KANTER
Spain, [7] [8]
Nail.
Proc.
Proc.
Neural
preprint
and
J.
ZIPSER
T. B.
GUTFREUND
and
1988).
Mass.,
1
S.,
KNUDSEN
D., in
Parallel
Nucl.
Acids
Distributed
Res.
18
Processing
(1990) (MIT
4787,
Press,
Cambridge,
;
Network J., Neural P., The Modeling of
E., J. Phys. W., MtzARD
Net.
;
and
lsl-193
Cambridge
(M.I.T. Press, (1990) 179.
Perceptrons
Trans.
ENGELBRECHT D. E.
pp.
S.,
PAPERT IEEE
(1988)
All M.
Anon
and L.
H., 1990, private
(van
Architectures Neural
Nostrand
(Cambridge
networks
Reinhold, Univ.
New
Press,
257.
NADAL
J.-P.,
F., J. Phys.
Complex Systems 1 (1988) 387 49 (1988) 1657.
France
communication
and
in
preparation.
1990) Chap.
York,
1991)
;
BURNOD
Y.
6.
