II. BIOMIMETIC SYNAPTIC CIRCUIT SYNTHESIS AND for a few exceptions) ionic current flow paths which enable. ANALYSIS the transmission of depolarising ...
A Biomimetic CMOS Synapse E.Lazaridis, E.M.Drakakis Member IEEE, M.Barahona Dept. of Bioengineering Faculty of Engineering - Imperial College London South Kensington Campus, SW7 2AZ, London, UK
{e.lazaridis, e.drakakis, m.barahona}gimperial.ac.uk Abstract-This paper presents the synthesis, analysis and basic
Clearly, a synapse is a critical element in the processing
operation of a new VLSI-compatible CMOS current-mode
of information in VLSI neural networks. There have been
synaptic circuit which implements the basic kinetics of a general chemical synapse in log-domain.
various different circuit implementations of artificial synapses. Most of them constitute a heavily or lightly approximated model of the biological synapse operation according to the computational context in mind. Contrary to those approaches, in this paper we focus upon the basic kinetics of a general chemical synapse and we present how a simple current-mode log-domain VLSI-compatible circuit topology can mimic them. Representative simulation results illustrating the proposed circuit's operation are provided. II. BIOMIMETIC SYNAPTIC CIRCUIT SYNTHESIS AND ANALYSIS
I. INTRODUCTION The term synapse introduced by Sherrington in 1900 describes the "specialised zone of contact at which one neuron communicates with another". In biological neural networks two basic types of synapses are encountered: electrical and chemical ones. Electrical synapses can be considered as physically present, ohmic, bidirectional (save for a few exceptions) ionic current flow paths which enable the transmission of depolarising signals from one neuron to another. They are characterised by virtually absent synaptic delay between the pre- and post-synaptic excitatory potentials. Nature seems to exploit electrical synapses for "fast" (e.g. Mauthner 's cell for the case of goldfish tail flip response) and "all-or-none"-type collective bursting responses (e.g. Aplysia's ink release). Contrary to electrical ones' the chemical synapses' operation does not rely upon the direct contact between the two neurons which are separated by a small space. This space is termed synaptic cleft and ranges from 20-40h In general chemical synapses mediate excitatory or inhibitory post-synaptic actions and can amplify signals by means of the following physicochemical mechanism: the pre-synaptic action potential initiates the release of chemical (neuro) transmitters stored in synaptic vesicles at the pre-synaptic terminal into the cleft, a procedure termed exocytosis. The neurotransmitters diffuse across the synaptic cleft and bind to post-synaptic receptors which de- or hyper-polarise the post-synaptic cell. Chemical synapses are unidirectional and are characterised by a significant synaptic delay of at least 0.3ms though it usually ranges between 1-5ms. In the vertebrate brain excitatory synaptic action is mediated by glutamate-gated channels whereas inhibitory synaptic action is usually mediated by y-aminobutyric acid (GABA) and glycine-gated post-synaptic channels. Several other neurotransmitters exist [1].
0-7803-9390-2/06/$20.00 ©2006 IEEE
A.
Synthesis
The synaptic conductance used for the computation ofthe synaptic current I is (usually) timesu f varying. In [2] a-functions were suggested for the modelling of neurotransmitter release. Such functions aim at the approximate preservation of phenomenological characteristics, namely physiologically recorded waveforms, and are deprived of any real connection with the underlying biological mechanisms. Such a bio-plausible connection is to a certain extent ensured by resorting to the computation of synaptic conductances by means of kinetic models which codify mathematically the markovian dynamics of neurotransmitter "release-binding to receptors" process. Such a model was proposed in [3] and was further simplified in [4]. Note that the adoption of a kinetic model for synaptic dynamics preserves (at least to a certain extent) both biomimicry and generality since such models apply widely to ion channel gating dynamics. In what follows we show how a rather simple current-mode logarithmic topology can realise a first-order kinetic synapse model. The synapse kinetics can be described by means of the following first-order differential equation [3,4]: r(t) =a[T](I - r) (l.a)
-f8r
751
ISCAS 2006
Readers well-versed towards the log-domain literature [5] would easily identify the topology of Fig. 1 as the skeleton neurotrasmitters The quantitisa dnotetcircuit of the lossy logarithmic integrator. However, it must be underlined that the circuit is not driven at all as a typical aneuro wanirs. rate qunsantities denote'tekforward first-order log-domain integrator where only a single and backward rate constants of thea,essynapse's kinetic current input is logged at Ml . Here two inputs Iinl and scheme: 'in2 are applied to the circuit. The particular derivation of R+T TR (2)
where r(t) denotes the time-dependent fraction of bound post-synaptic receptors and [T] denotes the concentration of
R represents the unbound whereas TR represents the bound receptors. A VLSI-compatible biomimetic synapse realising the dynamics (la) can be synthesised. More specifically: Re-expressing (l.a) as:
r(t)+ (a[T] +/3) r(t) =a[T]
(3.a-c) type relations can be found in [6]; it is not within the scope of this paper to derive the general dynamics of the topology of Fig. 1. We simply stress that the derivation of (3.a,b) does not rely on any small-signal linearisations; they describe the dynamics of the topology in Fig. 1 in a "largesignal" sense. Re-expressing (3.a) as . 2 I (5) w(t)+0 in2 w(t)= ifl
IF
(Lb)
we can directly see the analogy between (l.b) and the general dynamics governing the CMOS weak-inversion topology of Fig. 1:
nCVTw.o+I
and
nCVT w(t)+I,n2 w(t) = iinl (t) Iout (t) = Io w(t)
(3.a), (3.b)
with
'l (t) e=p w(t) n=
(3.c)
F Vcap (t) ]j
i(t)j LVnT
L
(a[T] +)6)
In (3.a,b) n corresponds to the MOS sub-threshold slope
LnVGS1 VTI
US9 IZT
lIn l
I
(4)
10
M2
MAlM
h
M3
+
ourt
hM4 1
1 54
I
(6.a)
~nCVT
6b
inl
(6.c)
nCVT Bearing in mind that a[T], ,8 have dimensions of rate we note that the RHS circuit quantities m (6.b,c) preserve the dimensions of the LHS biological quantities also present in (6.b,c). Moreover, the variables r(t) (fraction of bound receptors) and w(t) (ratio of two currents) are both We can re-express (6.b,c) as:
~~~~~~~~~~~~~~~dimensionless.
Iin2 oc nCVT (a[T]+ /3) > 'M2 '(nCVT/8) + (nCVTa[Tb = 1A + yaT
(7.a)
and (7.b) 'inl oc nCVT a[T] = a,T where I8 and IaT are quantities with dimensions of current and values dependent upon the biological forward and backward rates a, /3. Hence, (7.a,b) codify the necessary conditions for the circuit of Fig. 1 to mimic in a biologically realistic manner the synapse's kinetics shown in (l.a,b).
Note that no constraint has been applied with respect to the release profile of the neurotransmitters. Usually their release profile is assumed to be of pulsatile shape: after a 19pre-synaptic action potential a pulse of neurotransmitters M 3with value is assumed to be released. That assumption apart from suggesting that IaT could be a pulse it can also lead to a closed-form expression for r(t) [4,7].
[T]Imax
fi
I + e v | Fig. 1 The biomimetic CMOS synapse circuit
-
Wlo T
a[T]
and
The devices are of equal aspect ratios and operate in the sub-threshold regime in saturation (VDS >> 4VT) and with VBS = 0 (an option allowed by the technology). These conditions lead to an exponential device characteristic of the form: ID = IW DO expF )
nCVT
r(t) -c w(t) (aT 6 cin2(6b
parameter value of the particular process used, C is the capacitor value, VT corresponds to the thermal voltage T 2,' and IOtu are the input and output currents respectively whereas Imn2 denotes the total second input current sourced from transistor M2; Io is the value of the0 dc current-source driving the diodeconnected device M3 which acts as a level-shifter of the capacitor voltage variations. The quantity w(t) equals Q ) with I(t) denoting the total drain current of M2 IJnl((
nCVT
and comparing with (1 .b), a direct one-to-one analogy can be drawn. The circuit in Fig. 1 can mimic the biological kinetics (lb) when
However, for the proposed synapse circuit
'aT
does not
to comply with such a constraint; other release profiles ~~~~~~~~~~~have could also be used. Furthermore, it would be useful to stress
752
that I,, and IaT can be set independent one of the other with a single physiological rate constant related to each of them (see (7.a,b)). With at least two orders of current magnitude accommodated in the sub-threshold regime, the orthogonal setting of IA and IaT certainly constitutes an additional useful feature of the proposed synapse allowing both flexibility and variety in the type of kinetics that could be realised. The output current IOUt given by (3.b) affects the synaptic current IYn (t) which is in general given by [4]:
Vcap (t = 0) denoting the capacitor voltage prior to the application of the pulse. An approximate (i.e neglecting transistor non-idealities) estimation would show that Vcap (t = 0) relates to transistor sizing and technologydependent parameters. For the output current 'out (I) will
hold:
Fw + _o, (t(t)
L
(8) syn(t) =gsynr(t)PSV(t)-syn J withgs1,n denoting the maximum conductance of the synapse, PSV(t) the post-synaptic potential and E the synaptic reversal potential. Because of (3.b), the current
Iout (t) -c w(t) -c r(t) Whence, isyn (t) .
-
r(t)
-
°#
which leads to ( IT
I8 +A +CVcap (t)
exp
vcap (t)
Vc
After some elementary algebraic manipulation: 1I3 + A A X (t)+ LnCV X(t) = nCV
l+A nCVTI (9.f).
I# can take values independent one of the other, a
Lg
A and
(9.b)
variety of exponentially rising and decaying dynamics (i.e. synapse characteristics) can be realised. As a concluding remark it is worth pointing that the general relation (5) coincides with (9.c) when (7.a,b) are taken into consideration; furthermore, (9.f) is equivalent to (3.b). 111. SIMULATION RESULTS the Adopting practice encountered in other publications [7,8], in this section we present simulation results of the proposed biomimetic synapse when excited both by individual pulses and trains of pulses. Figs. 2 and 3 illustrate the circuit's response (i.e.Iout(t)) when IaT corresponds to a single current pulse of 0.2ms duration. In Fig.2 the pulse amplitude is 0.5nA whereas in Fig. 3 it is 5nA. The graphs illustrate the resulting I t (t) waveforms for two different values of current IP, namely 0.1 and 0.2 nA; the higher the
(9.c)
where the variable X(t) is related to the time-dependent capacitor voltage as follows: V (t)= p n VT ( The solution of (9.c) is given by: I ±A ] A +FA(I +A) (9.e). CV A nCVTj + X(t n)exPL X(t = LfnCVTj The time variable t refers to the IaT pulse duration with the value t = 0 corresponding to the instant that the 'aT pulse is applied. The quantity X(t = 0) ocexp(K ap(t )= with
nCV
speed of the exponential decay of Iout . Bearing in mind that
(9.a)
A):
(9.f
A3+A11
with tend denotmg the mstance at which the IT pulse stops to apply. Now the time constant nCVT determines the
flVT
DOYL)F
+A
!K
XV (t)) nVT
it becomes clear that the time constant nCVT +A I,t (t) + A determines the speed by which the output current amplitude rises exponentially after the application of the IT pulse. Based on (9.c) it is a simple matter to show that after the end of the applied pulse the output current will decrease exponentially according to the scheme: (9.g) lout (t - tend) out (tend) exp CV(t - tend
From
w(t)
with the current Io clearly affecting the strength of the synaptic current since I'ot (t) = Io w(t) . (A detailed treatment of a small circuit realising (8) will be presented elsewhere). B. Analysis Assuming that the applied current IaT has the shape of an ideal pulse of amplitude A and applying KCL at the capacitor node of Fig. 1 yields: = fvT lfKIaT /LIDO -I Vpw L 1/ +IaT +CVcap(t) l ( I
10A
w ( LIDO L)) nfVT
Ip value, the faster Iou (t) decays after the end ofthe 'aT pulse as expected from (9.g). Fig. 4 confirms how the
flVT )value ofthe current Io affects the strength ofthe synapse; 753
when I takes increasing values of 1,2 and 3 nAs, the plateau of Io increases as expected from (9.f) (and (3.b)); check that since I, remains constant, the exponential decay of lIt after the end of the applied pulse is governed by the same time constant. Fig.5 illustrates the synapse's response , zs * fsfs . is to two trains of pulses of different amplitude; each pulse of 20Os duration; the duration of a "burst" of pulses is 160s followed by a "quiet" period of 1 Ops. The circuit achieves the (exponential) summation of incoming spikes; stronger pulses lead to higher overall output current. During the "quiet" period the output decays exponentially as expected.
dnmpl=010pVIPLUS eOOP
400D 20o ddrr p l=-1l00p- 2'/P LUS 0h ~~~~~~~~~~~~~~2
0 l.3n
WX
6
-3m
3r Mm5
43m 4.;3m
44.7m
Fig.2 Single pulse synapse response(bottom); dFi p12ing 0pusna0/PUJS =: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~d,qmp1=`V00p";/VS1,/PLUS
*
5.1 rr
5.5n
saT 0.5nA(top)
4.0.n
In all cases the synapse operated from a 3V power supply; however, it was confirmed that the circuit could operate with lower power supply levels. All devices had an aspect ratio of lIm/lIm. Process parameters from a commercially available 0.8 gm technology were used. Numerous simulations have been performed which confirm that the circuit is suitable for implementing a variety of synapses. Here we present only a few representative results. Table 1 summarises the basic circuit parameters used for the collection of the graphs shown in Figs. 2-5. The correlation of these circuit quantities with the biologically relevant forward and backward rate constants a,,8/ is straightforward by means of (7.a,b). Table 1 also provides the / rate constants corresponding to the presented circuit parameter value combinations.
2,On
00
4A47511
__;_______________-D_i_D-''_D'___I_____'_'_''_'_'_'_ dp1=`100p-,/2/PLUS
2.950r
lI425n
4 5m 5m Jim ( 5I) 47m Fig.3 Single pulse synapse response(bottom); IaT-5nA(top) _
3
a;V/PLUS5
__ __
r
__
_ _
='2;V/LJ
__p_ _
_
_
REFERENCES /V2/PLUS ________ ro in /2L =:_ rM=2n;A2/PUJS [1] E.R.Kandel, J.H. Schwartz and T.H.Jessell, "Principles ofNeural Science", 4th -Edition, Mc-Graw Hill, 1991 [2] W.Rall, "Distinguishing theoretical synaptic potentials computed for _ _ different somadendritic distributions of synaptic inputs", Journal of _4_0 Neurophysiology, vol.30, pp. 1138-1168, 1967 [3] D.H.Perkel, B.Mulloney and R.W.Budelli, "Quantitative methods for ________ predicting neuronal behaviour", Neuroscience, vol. 6, pp. 823-827, 1981 m 5.m 43mt 5:1m [4] A.Destexthe, Z.F.Mainen and T.J.Sejnowski, "An efficient method 4s 7 tie5m for computing synaptic conductances based on a kinetic model of receptor Fig.4 Synapse strength is affected by Io; lo=l, 2, 3 nAs(bottom); binding", Neural Computation, vol. 6, pp. 14-18, 1994 IaT0.5nA(top) [5] D.R.Frey, "Exponential State-Space Filters: a Generic current-mode 2.0n a=U20p;A,PLUS Design Strategy", IEEE Trans. CAS-I, vol.43, No.1, pp.34-42, 1996 [6] E.M.Drakakis et al. "Log-domain state-space: a systematic transistor level approach for Log-domain filtering", IEEE Trans. On CAS-Part II, vol. 10 46, no.3, pp.209-305, 1999 500F [7] R.Z.Shi and T.Horiuchi, "A summating, exponentially decaying re-
CMOS synapse for spiking neural networks", 1999 [8] S-C Liu, "Analog VLSI circuits for short-term dynamic synapses",
M=l-iV2/PLUS
Eurasip journal on Applied Signal Processing, pp.620-628, 2003
Figure 3 4
5
C(pF)
lo(nA)
lp(nA)
0.4
1
0.1-0.2
f3(msec-') [tt(gs)] 7.4/14.8
[67.6)/135.2]
-47
1 Y-Y-lm - 22m 0.4I~~~~~~~~~~~~~~~~~~~~~~~nI 1 0.1-0.2 7.4/14.8 [67.6/135.2] 7;.0 .lrl 2nL |0.4 1-3 |0.05 3.7 [270] Fig.5SSynapse response to trains of pulses (see text for numerical details) 2
1
|0.05
0.74 [1.35ms]
Table 1 Synaptic circuit parameters
754
