network complexity and synchronous behavior - Semantic Scholar

June 3, 2010 12:3 00239

International Journal of Neural Systems, Vol. 20, No. 3 (2010) 233–247 c World Scientific Publishing Company DOI: 10.1142/S0129065710002395

NETWORK COMPLEXITY AND SYNCHRONOUS BEHAVIOR — AN EXPERIMENTAL APPROACH P. J. NEEFS ASML — DE IVS Immers. & EUV Overlay De Run 6501, 5504 DR Veldhoven, The Netherlands [email protected] E. STEUR∗ and H. NIJMEIJER† Dept. of Mechanical Engineering, Eindhoven University of Technology P.O. Box 513 5600 MB, Eindhoven, The Netherlands ∗ [email protected] † [email protected] We discuss synchronization in networks of Hindmarsh-Rose neurons that are interconnected via gap junctions, also known as electrical synapses. We present theoretical results for interactions without timedelay. These results are supported by experiments with a setup consisting of sixteen electronic equivalents of the Hindmarsh-Rose neuron. We show experimental results of networks where time-delay on the interaction is taken into account. We discuss in particular the influence of the network topology on the synchronization. Keywords: Network synchronization; neuronal cell; time-delay; electronic brain.

1. Introduction

that will lead to such behavior, exploring the possibilities to manipulate these conditions, and describing them rigorously is vital for further progress in neuroscience and related branches of physics. This paper focusses on the behavior of a network of identical cells, including the possible emergence of a simultaneous synchronized state. Synchronization is investigated for a few connected systems. e.g. see (Ref. 8), however, in the last decades research has shown that the complexity of the network configuration appears to affect its synchronous behavior, see for instance (Refs. 9 and 10). The prototype network considered will be a network of HindmarshRose neuronal oscillators.11 In particular we will present results on synchronization in ensembles of Hindmarsh-Rose neuronal oscillators which are being interconnected via gap-junctions, also known as electrical synapses. Recently it has been pointed out that gap-junctions play an important role in synchronization of individual neurons.12 Gap-junctions can be modeled as linear electrical couplings of the form I12 (t) = γ · (v1 (t) − v2 (t)) where I12 is the synaptic

Synchronization is the phenomenon where systems, due to some kind of interaction, adjust their individual behavior in such a way that their behaviors become identical. Remarkably synchrony is often encountered in biological systems. Well known examples include the simultaneous flashing of fireflies,1 the synchronous activity of pacemaker cells in the heart2 and synchronized bursts of individual pancreatic β-cells.3 Many more examples can be found in, for instance, (Refs. 4 and 5) and the references therein. Besides in biological systems synchrony is also witnessed in network of neurons. Neurons in the brain discharge their action potentials in synchrony for various reasons. In fact, synchronous oscillations of neurons have been reported in the olfactory bulb, the visual cortex, the hippocampus and in the motor cortex.6,7 The presence or the absence of synchrony in the brain is often linked to specific brain function or critical physiological state (e.g. epilepsy). Hence, understanding the conditions †

Corresponding author. 233

June 3, 2010 12:3 00239

234

P. J. Neefs, E. Steur & H. Nijmeijer

current, the constant γ represents the synaptic conductance and v1 (t) − v2 (t) denotes the difference in membrane potential of the neurons at the presynaptic side and the post-synaptic side at time t, respectively. The interaction will be mutual, that is, if neuron 1 is influenced by neuron 2 through the synaptic current I12 (t), then neuron 2 will be influenced by neuron 1 by the current I21 (t) = −I12 (t). Hereby it is assumed that the propagation speed of the signals through the axon and/or dendrites is infinite. In case of a finite propagation speed the gapjunctions can be modeled via the linear time-delayed coupling I12 (t) = γ · (v1 (t − τ ) − v2 (t − τ )), where τ > 0 denotes a time-delay which is induced by the finite propagation speed. In particular, we will present a general framework with which we can guarantee synchronization in a network of identical Hindmarsh-Rose neurons with non-time-delayed interaction under the restriction that the synaptic conductance, in the sequel referred to as the coupling strength, is sufficiently large. This result, in combination with the Wu-Chua conjecture,13 does not only guarantee synchrony but also provides insight in how the topology of the network influences the synchronous behavior. We will demonstrate synchronization in an experimental setup that confirm the theoretical results. The experimental setup consists of sixteen electronic equivalents of the Hindmarsh-Rose neuron. These electronic neurons interact with each other via a custom build synchronization interface in which the network topology and the interaction strengths can be defined. Besides that such an experimental setup can be used to verify theoretical results, the use of it also provides important insight in how robust the synchronization is, i.e. in such a setup the individual systems will never be exactly identical and those systems operate in a noisy environment. Furthermore, using this experimental setup we investigate the effect of the network topology in the case where time-delay is present in the coupling, i.e. we will present experimental results with coupling (3). Unfortunately rigorous theoretical results for this type of interaction are, to the best of our knowledge, not available in literature. However, our experimental results indicate again that there again exists a Wu-Chua like conjecture for this type of coupling. The paper is organized as follows. In Sec. 2, we formulate the problem. Then, in Sec. 3, we discuss

the experimental setup. In Sec. 4, we discuss the nontime-delayed coupling. In particular we introduce the theoretical framework and we present the experimental results. The experimental results for the timedelayed coupling will be presented in Sec. 5. In Sec. 6, we summarize the results and we give suggestions for further research. 2.

Problem Statement

In this section we formally introduce the problem setting. We consider networks of Hindmarsh-Rose neurons v˙ i (t) = −c1 vi3 (t) + c2 vi2 (t) + c3 vi (t) − c4 + c5 w1,i (t) − c6 w2,i (t) (1) + c7 I + c8 u(t), w˙ 1,i (t) = −c9 vi2 (t) − c10 vi (t) − c11 w1,i (t), w˙ 2,i (t) = c12 (c13 vi (t) + c14 − w2,i (t)), where ˙ := dtd∗ , t∗ = 1000t, i = 1, 2, . . . , n, 2 ≤ n ≤ 16, vi denotes the membrane potential of neuron i, which also serves as the natural output of the neuron, w1 , w2 are internal variables, input u(t), constant parameters c1 , . . . , c14 and bifurcation parameter I. Depending on the value of I the Hindmarsh-Rose neuron (1) is in its resting state, i.e. a constant (negative) membrane potential, (2) produces (chaotic) bursts, or (3) operates in the spiking regime, cf. (Ref. 11). We will use a constant I = 3.3 which lets the neuron operate in the chaotic bursting regime. In the sequel we will use the parameters presented in Table 1. With these parameters we will be able to build an electrical equivalent of the Hindmarsh-Rose neuron (1) using off-the-shelf components. The neurons (1) interact via gap-junctions which will be modeled as n γij (vi (t) − vj (t)), (2) ui (t) = − j=1,j=i

where the synaptic conductance γij = γji ≥ 0. The conductance γij = 0 if and only if there is no direct connection between neurons i and j. In case we take time-delay into account the coupling is given by n γij (vi (t − τ ) − vj (t − τ )), (3) ui (t) = − j=1,j=i

with time delay τ > 0.

June 3, 2010 12:3 00239

Network Complexity and Synchronous Behavior

Table 1. Nominal parameter values of the Hindmarsh-Rose neural model (1). c1 = 1, c5 = 5, c9 = 1, c13 = 4,

c2 = 0, c6 = 1, c10 = 2, c14 = 4.472.

c3 = 3, c7 = 1, c11 = 1,

c4 = 8, c8 = 1, c12 = 0.005,

235

neurons interact via coupling (2) or (3). The neurons are called asymptotically synchronized, or simply synchronized, if for any initial conditionsa the following asymptotic relation holds xi (t) − xj (t) → 0,

as t → ∞,

(7)

where xi (t) := col(vi (t), w1,i (t), w2,i (t)). Define the n × n coupling matrix   n −γ12 ... −γ1n j=2 γ1j n  −γ ... −γ2n    21 j=1,j=1 γ2j , Γ= .. .. .. ..   . . . .   n−1 −γn1 −γn2 ... γ nj j=1 (4) which allows to write the couplings (2) and (3) as u(t) = −Γv(t),

u(t) = −Γv(t − τ ),

(5)

respectively, where u(t) = col(u1 (t), . . . , un (t)) and v(t) = col(v1 (t), . . . , vn (t)). Here the notation col(x1 , . . . , xn ) stands for the column vector composed of the vectors x1 , . . . , xn . The coupling matrix appears to be very useful in the study of synchronization since it contains the necessary information of the network topology and the interaction strengths, cf. (Refs. 13 and 14). We summarize some of the properties of Γ. • The coupling matrix Γ is symmetric by construction, i.e. Γ = ΓT , where the superscript T denotes transposition. Since Γ is symmetric its eigenvalues are real. • The coupling matrix Γ is singular since all rows sum to zero. We will assume that the network cannot be decomposed into two or more disjoint networks. It is well known that under this assumption the zero eigenvalue of Γ is simple. • Using Gerschgorin’s theorem one can easily verify that Γ is positive semi-definite. Hence we can order the eigenvalues λi of the coupling matrix Γ as 0 = λ1 < λ2 ≤ · · · ≤ λn .

(6)

Let us formally define the notion of synchronization. Definition 1 (Synchronization). Consider a network of n Hindmarsh-Rose neurons (1) and let the a

In the remainder of this paper we present sufficient conditions for the neurons to synchronize in the sense of Definition 1 for the non-time-delayed coupling and we show experimental results for both nontime-delayed and time-delayed interaction. We will, for notational convenience, often drop the explicit dependency on time of the variables, i.e. x(t) will simply be denoted by x. 3.

The Experimental Setup

In this section we introduce the experimental setup, which consists of sixteen (non-identical) electronic Hindmarsh-Rose neural oscillators and a coupling interface. 3.1. The electronic Hindmarsh-Rose neuron By applying Kirchoff’s laws and the well-known properties of operational amplifiers we have derived an electronic circuit which governs the HindmarshRose equations(1). This realization is partly based on the circuit presented in (Ref. 15). The circuit basically consists of three integrator circuits, which integrate the three states of the Hindmarsh-Rose model (1), respectively. See Fig. 1. A multiplier circuit, build around two AD633 analog multipliers, is implemented that generates the squared and cubic term that are present in this model. Using Ohm’s law and Kirchhoff’s rules one can easily verify that the circuits depicted in Fig. 1 represent the Eq. (1). The states vi , w1,i and w2,i can be directly measured as a voltage. In addition, the circuit has two voltage inputs, namely I and ui . Note that in the original equations (1) the inputs I and ui actually represent currents, however, in an experimental setup it is more convenient to use voltage inputs. The realization of this electronic equivalent is depicted in Fig. 2.

In case that the neurons interact via the time-delayed coupling (3) we have to replace the initial conditions by initial function segments defined on the interval [−τ, 0].

June 3, 2010 12:3 00239

236


Fig. 2.

The electronic Hindmarsh-Rose neuron.

1 0 −1 −2 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 0.5 0 −0.5 −1 −2.4 −2.6 −2.8 −3 −3.2

Fig. 3. State trajectories of an electronic HindmarshRose neural oscillator.

I = 3.3[V]. These measured signals are very similar (almost identical) to the signals that one can obtain by numerical integration of the Eqs. (1). Here mean of course that the shapes of the spikes, timing and the ranges of the signals are almost identical since we cannot expect the experiment and numerical simulation exactly coincide due to the chaotic behavior of the system, (small) numerical errors and slight imperfections in the realization. Fig. 1. The circuits corresponding to the states of a Hindmarsh-Rose neuron.

3.2. The coupling interface

Experiments are performed with an electronic Hindmarsh-Rose neuron to investigate its dynamical performance. Figure 3 shows the measured chaotic outputs of the circuit for the applied (constant) input

The sixteen electronic Hindmarsh-Rose neurons interact via a coupling interface in which the network topology and the coupling strengths can be defined. This electronic device, schematically illustrated in Fig. 4, contains sixteen (voltage) input

June 3, 2010 12:3 00239


Fig. 5.

4. Fig. 4. face.

Schematic representation of the coupling inter-

and (voltage) output channels. The inputs of the synchronization interface are connected to the outputs vi of the neurons, and the outputs of the interface are the inputs ui of the neuron. The interface samples the membrane potential of all connected electronic neurons simultaneously. An ARM9 micro-controller computes the coupling functions. In particular, the network topology and coupling strengths are defined in c++ -code which is loaded into the micro-controller. Sixteen Digital Analog Converters (DACs) convert the digital signals into the analog signals that are the inputs of the electronic neurons. Like the sampling the update of the DACs is simultaneous as well. As a result of the inevitable delay induced by the data acquisition, the data processing and the update is the same for each input-output channel. We estimated this delay at maximal 80[µs] (in case we couple all sixteen systems). The sample rate of the coupling interface is about 40 [kHz] when all sixteen channels are used. An advantage of defining the coupling functions using a dedicated micro-controller is that we can in principle implement any type of coupling, e.g. linear non-delayed coupling, linear delayed coupling and nonlinear coupling. For convenient experimentation two stacks, each capable to contain up to ten electronic neurons, have been realized. Four SigLab data-acquisition devices that sample simultaneously are used to measure the membrane potential of the sixteen neurons. A picture of the complete experimental setup is presented in Fig. 5.

237

The complete experimental setup.

Non-Time-Delayed Interaction

In this section the results for the non-time-delayed interaction of the Hindmarsh-Rose neurons are presented, i.e. we consider the systems (1) that interact via the coupling (2): ui = −

n

γij (vi − vj ),

γij = γji ≥ 0.

(8)

j=1,j=i

First it will be proved that these neurons indeed synchronize given that the coupling is sufficiently strong for a given network topology. We also present the Wu-Chua conjecture that directly allows to address the influence of the network topology. In the end we present experimental results. 4.1. Theoretical framework We will use a passivity based framework, introduced in (Refs. 14,16), which provides sufficient conditions for synchronization in the sense of Definition 1. First we introduce the notion of semi-passive systems. Definition 2 (Passivity and Semi-passivity14,17). Consider the system x˙ = f (x, u),

y = h(x),

(9)

where state x ∈ Rp , input u ∈ Rm , output y ∈ Rm and sufficiently smooth functions f : Rp × Rm → Rp , h : Rp → Rm . The system (9) is said to be passive if there exists a nonnegative storage function V : Rp → R+ , V (0) = 0, such that the following dissipation inequality ∂V (x) V˙ (x) = (f (x, u)) ≤ y T u (10) ∂x holds. If inequality (10) is satisfied only for x lying outside some ball, i.e. V˙ (x) ≤ y T u − H(x),

(11)

June 3, 2010 12:3 00239

238


where the function H : Rp → R is nonnegative outside some ball ∃ ρ > 0,

x ≥ ρ ⇒ H(x) ≥ (x),

(12)

with some nonnegative continuous function (·) defined for x ≥ ρ, then the system (9) is called semi-passive. If the function H(·) is positive outside some ball, then the system (9) is said to be strictly semi-passive. A semi-passive system behaves as a passive system for large enough x. Hence, the physical interpretation that goes along with a semi-passive system is that of a system with a finite amount of “free” energy delivered by some internal source. As proven in (Ref. 14), a network of strictly semi-passive systems interconnected by the coupling (2) has ultimately bounded solutions, that is, regardless of the chosen initial conditions, each solution of the closedloop system enters a compact set in finite time and remains there as time increases. Proposition 1. The Hindmarsh-Rose neuron (1) with output vi , input ui and the parameters given in Table 1 is strictly semi-passive. The proof of Proposition 1 can be found in the appendix. The following theorem, adopted from (Ref. 14), gives sufficient conditions for synchronization of systems which interact via a coupling of the form (2). Consider n coupled systems of the

Theorem 1. form

y˙ i = a(yi , zi ) + ui ,

(13)

z˙i = q(zi , yi ),

where i = 1, . . . , n, output yi ∈ Rm , input ui ∈ Rm , internal variable zi ∈ Rp−m and sufficiently smooth functions a : Rm × Rp−m → Rm , q : Rp−m × Rm → Rp−m . Let the systems interact via coupling (2). Suppose that (1) each system is strictly semi-passive with a radially unbounded storage function; (2) there exists a C 2 -smooth positive definite function V0 : Rp−m → R+ , V0 (0) = 0, such that ∇V0T (z − z ∗ )(q(z , y ) − q(z ∗ , y )) ≤ −αz − z ∗ 2 ,

for all z , z α > 0.

∗

∈ R

p−m

, y ∈ R

m

(14)

with constant

Then the solutions of the closed loop system (13), (2) are ultimately bounded. Let the eigenvalues of the coupling matrix Γ be given as 0 = λ1 < λ2 ≤ · · · ≤ λn ,

(15)

¯ such that if λ2 ≥ λ ¯ the then there exists a number λ systems (13) synchronize in the sense of Definition 1. The theorem can be explained as follows. Given that the systems satisfy both conditions of the theorem, then if the coupling is strong enough for a network with a given topology, i.e. the smallest nonzero eigenvalue λ2 of the coupling matrix is large enough, then the systems will synchronize. One might have noticed that the Hindmarsh-Rose neuron is already in the form (13). Furthermore, from Proposition 1 it is known that the Hindmarsh-Rose neuron satisfies condition 1 of the theorem. To show that condition 2 is also satisfied for the HindmarshRose neuron we will introduce the notion of convergent systems. Definition 3 (Convergent Consider the system

systems18,19 ).

z˙ = q(z, ω(t)),

(16)

where state z ∈ Rp−m , an external signal ω(t) which takes values from a compact set Ω ⊂ Rm and a sufficiently smooth function q : Rp−m × Ω → Rp−m . The system (16) is called convergent if • all solutions z(t) are well-defined for all t ∈ (−∞, +∞) and all initial conditions z(0), • there exists an unique globally asymptotically stable solution zω (t), which depends on the driving input ω(t), defined on the interval t ∈ (−∞, +∞) from which it follows lim z(t) − zω (t) = 0

t→∞

for all initial conditions. The long term motion of systems of this type is solely determined by the driving input ω(t) and not by initial conditions z(0). A convergent system “forgets” its initial conditions. A sufficient condition for a system to be convergent is given in the next lemma.

June 3, 2010 12:3 00239


Lemma 1. 18,19 If there exists a positive definite symmetric (p − m) × (p − m) matrix P such that all eigenvalues λi (Q(z, w)) of the symmetric matrix

1 ∂q ∂q Q(z, w) = P P (z, w) + (z, w) 2 ∂z ∂z (17) are negative and separated from zero, i.e. there is a δ > 0 such that λj (Q(z, w)) ≤ −δ < 0,

(18)

with j = 1, . . . , p − m for all z ∈ Rp−m , ω ∈ Ω, then the system (16) is convergent. It follows from the definition of a convergent system that if there exists such a matrix P such that each system z˙i = q(zi , yi ) satisfies (17), (18), i.e. each system z˙i = q(zi , yi ) is convergent, then there exists a positive definite function V0 that satisfies inequality (14) of Theorem 1. To show that the Hindmarsh-Rose neurons will synchronize whenever the coupling is strong enough we now only have to prove that the wi -dynamics of the neuron are convergent. Proposition 2. The internal dynamics of a Hindmarsh-Rose neuron, i.e. the wi -dynamics are convergent. The proof follows immediately from Lemma 1 with, for instance, P = I2 , where I2 denotes the 2 ×2 identity matrix. Since both conditions of Theorem 1 are satisfied a network of Hindmarsh-Rose neurons (1) which interact via coupling (2) will synchronize if the coupling is sufficiently strong. There are several methods to determine/estimate ¯ One might estimate the value from the threshold λ. direct computation (using a Lyapunov for the error system) or the threshold can be determined via the Master Stability Function (MSF).20 Note one has to be careful when applying the latter since the MSF only gives conditions for local synchrony. We ¯ with the help of estimated the threshold value λ b Simulations c

239

numerical simulations of two coupled neurons.b It is ¯ ≈ 1. found that the threshold for synchronization λ Using a conjecture posed in (Ref. 13) we can predict how strong the coupling should be in a network with a different topology to achieve synchronization. Conjecture 1 (Wu-Chua conjecture13 ). Suppose that the neurons in a network with coupling matrix Γ1 synchronize, then the neurons in a network with coupling matrix Γ2 synchronize if and only if λ2 (Γ1 ) = λ2 (Γ2 ),

(19)

that is, the smallest nonzero eigenvalue of Γ2 equals the smallest nonzero eigenvalue of Γ1 . Although the Wu-Chua conjecture is not true in general, for systems that satisfy the conditions of Theorem 1 the conjecture holds.14 4.2. Experimental synchronization Using the machinery presented above we will now demonstrate synchronization of the electronic Hindmarsh-Rose neurons which interconnect via the non-time-delayed coupling (2). We present results of three different networks, namely an all-to-all coupled network, a network with two-nearest-neighbor coupling and a small-world network.c Hereby we assume the coupling strength to be uniform, i.e. if there is a connection between neurons i and j, then γij = γ. Otherwise γij = 0. However, we will first discuss the simplest case of only two coupled electronic Hindmarsh-Rose neurons. Because the systems in the experimental setup are nearly but not exactly identical, we cannot expect that the systems synchronize in the sense of Definition 1. We therefore introduce a slightly weaker notion of synchronization which we will refer to as practical synchronization. Definition 4 (Practical synchronization). Consider a network of n Hindmarsh-Rose neurons (1) and let the neurons interact via coupling (2) or (3). The neurons are called practically asymptotically synchronized, or simply practically synchronized, if for any initial conditionsd the following limiting relations

are performed with MATLAB-Simulink. A small-world network is characterized by a large clustering coefficient such as encountered in regular networks but at the same time the average pathlength is small, just as in random graphs. See (Ref. 21) for more details. d In case that the neurons interact via the time-delayed coupling (3) we have to replace the initial conditions by initial function segments defined on the interval [−τ, 0].

June 3, 2010 12:3 00239

240


holds lim sup xi (t) − xj (t) ≤ ε,

(20)

t→∞

where xi (t) := col(vi (t), w1,i (t), w2,i (t)) and a sufficiently small constant ε > 0. For practical reasons, i.e. a limited number of data acquisition channels, we will only measure the vi -state of the electronic neurons. Hence we apply Definition 4 only to the difference between the vi -states of the coupled systems (1). Nevertheless, due to the fact that the internal dynamics of the Hindmarsh-Rose neuron are convergent and smoothness of the vectorfields, practical synchronization of the vi -state straightforwardly implies synchronization of the internal states w1,i , w2,i . We say that the electronic Hindmarsh-Rose neurons practically synchronize when the conditions of Definition 4 are satisfied with ε = 0.25. Although the value ε = 0.25 seems to be rather high at first sight, one has to note that due to the spiking behavior of a neuron a small mismatch between these spikes might result in a relatively large error. The minimal coupling strength for which the neurons practically synchronize for a given network will be denoted by γ . Let us now consider two coupled neurons (1), n = 2. The experimental results can be found in Fig. 6. Clearly, after some transient behavior, the two neurons rapidly synchronize. Here we used coupling 1 0 −1 −2 −3 0

0.5

1

1.5

0.5

1

1.5

0.5

1

1.5

strength γ = 0.8. We determined the minimal coupling strength required for synchronization in the experimental setup to be γ ≈ 0.6, which is pretty close to the threshold γ¯ ≈ 0.5 that we estimated using simulations. (Note that the threshold of the smallest nonzero eigenvalue of the coupling matrix ¯ ≈ 1 which corresponds to the minimal coupling λ strength γ¯ ≈ 0.5.) At a simulation level, being the ideal situation with identical models, the error between the states of both systems converges to zero when synchronized. This implies that the diagonal v1 = v2 in the (v1 , v2 )-plane forms the unique attracting set for all state trajectories, see the dashed line in Fig. 7. Note that we hereby ignore the transient behavior. In the experiment however, the solutions will only approach this diagonal within the bound ε. See the solid line in Fig. 7. From this representation we can therefore immediately conclude whether the systems are practically synchronized or not by looking at the deviation of the measured signals from the diagonal. This, along with the ability of the used data-acquisition to obtain a real-time (vi , vj )-plane visualization, results in a fast analysis of the synchronous behavior of coupled systems in a network. We will now investigate synchronization of networks consisting of multiple systems. In particular, we treat the following three topologies formed by sixteen electronic neurons: • An all-to-all coupled network, • A two-nearest-neighbor (ring) topology, • A small-world21 network configuration.

1 0 −1 −2 0 −2 −3 0

Fig. 6. Experimental synchronization of two coupled electronic Hindmarsh-Rose neurons, γ = 0.8. The black thick line indicates the time at which the coupling became active.

Fig. 7.

Experimental synchronization boundary.

June 3, 2010 12:3 00239


Fig. 8. A 2-nearest-neighbor configuration (upper) and a small-world topology (lower).

For clarity -the first network being straightforwardthe second and third network are depicted in Fig. 8. Similar to the previous example the coupling strength necessary to synchronize all systems in the network is obtained experimentally. The results are presented in Table 2. In addition, a visualization of the synchronized states for the all-to-all coupled network configuration is presented in Fig. 9. The figures for the two-nearest-neighbor network and the small-world network will look similar, and therefore we omitted these figures. To illustrate the reliability of the experimental results a comparison is made with semi-analytical values. Table 2. Coupling strengths obtained for the networks (n = 16) considered in this section. Type All-to-all 2-nearest-neighbor Small-world

γ

0.065 1.6 1.3

γ [Wu-Chua] 0.0625 1.3350 1.1120

241

Obviously, the experimental results match closely to the ones obtained by the Wu-Chua conjecture. By this, the performance of the setup again appears to be satisfactory. From the results we can also conclude some aspects regarding the complications of the topological complexity on the synchronous behavior in a network. Clearly, the all-to-all connected topology has the shortest possible paths between the neurons, i.e. the network is maximally connected. Hence the smallest nonzero eigenvalue of the corresponding coupling matrix will be relatively high. In fact, in case of an all-to-all network λ2 = γn, that is, the smallest nonzero eigenvalue equal the number of systems in the network multiplied by the coupling strength. The connectivity of the two-nearest-neighbor network is much lower than that of the all-to-all network. Hence the corresponding smallest nonzero eigenvalue will be much lower such that a relatively large coupling strength is required to let the neurons end up in synchrony. In fact, in the case where the number of systems approaches infinity, n → ∞, one would require an infinitely strong coupling, γ → ∞ to achieve synchrony.14 Although the small-world network contains almost the same regularity as the two-nearest-neighbor topology, the randomly rewiring21 procedure of some edges yields a few sparse connections. The latter is responsible for a shorter (average) distance between two systems in the network that are separated the farthest away from each other. This results in a larger smallest nonzero eigenvalue of the corresponding coupling matrix compared to the one of the two-nearestneighbor configuration. To this end, the coupling needed to synchronize all systems can be lower than that of the 2-nearest neighbor topology. The results presented in Table 2 do confirm this. 5.

Time-Delayed Interaction

For our second set of experiments we will include time-delay on the interaction, i.e. the neurons will be coupled via coupling (3): ui (t) = −

n

γij (vi (t − τ ) − vj (t − τ )),

j=1,j=i

τ > 0. (21) Since, at the moment, there is a lack of rigorous mathematical results that provide sufficient

June 3, 2010 12:3 00239

242


Fig. 9.

Experimental synchronization results of n = 16 diffusively coupled systems in an all-to-all topology for γ = 0.1.

conditions for synchronization of time-delayed networks, we will only present some of the experimental results obtained along with the conclusions that can be drawn from those. (Most results in the literature are formulated in terms of Linear Matrix Inequalities (LMIs) which in general tend to be very conservative, cf. (Refs. 22 and 23).) First, we treat the case of only two coupled electronic neurons. Thereafter multiple coupled neurons are considered up to a maximum number of four. This maximum number results from the limitations of the synchronization interface as will be explained later on. Again, to reduce the complexity we assume a uniform coupling strength. 5.1. Experimental synchronization Different from our first set of experiments, due to two variable parameters (γ, τ ) that each influence the stability of the synchronization manifold, we have to investigate the synchronous behavior of time-delayed coupled network configurations in the 2d-diagram spanned by these parameters. For simplicity, we will first consider the simplest case of two coupled systems. This should make the reader familiar with certain aspects regarding the synchronization

of time-delayed coupled systems without being confused by topological aspects that appear to affect this behavior as we concluded from our previous results. For different combinations (γ, τ ) the experimental synchronization of the two coupled electronic neurons is investigated, the results are given in Fig. 10. The two systems appears to synchronize for a certain collection of parameter values, depicted by the grey area in the top figure. An illustrative example of this synchrony is given in the bottom figure for the point (γ, τ ) = (2, 0.5). From this result we, at least, can conclude the following: • The minimal coupling strength for which the systems synchronize equals that for τ = τmin ≈ 80[µs], see Remark 1. • There appears to exist a maximum time-delay, which we denote by τmax . Synchronization is definitely lost when τ > τmax . • For each time-delay τ that belongs to the interval [0, τmax ) there exist a range of coupling strengths for which the neurons will synchronize. This range is given by two functions γmin (τ ) and γmax (τ ), which provide a lower bound and an upper bound

June 3, 2010 12:3 00239


243

1

Fig. 11. Different time-delayed diffusively coupled networks Ge |e=1,2,...,7 .

0 −1 −2 −3 0

3 0.2

0.4

0.6

0.8

1

0.01

2.5

0

2 −0.01

1.5 −0.02

0

0.2

0.4

0.6

0.8

1

1

Fig. 10. Experimental results of two time-delayed diffusively coupled Hindmarsh and Rose oscillators. Asymptotic synchronization appears to occur in the grey area (top panel). An example of this synchrony is presented for γ = 2.0 and τ = 0.5[ms] (bottom panel).

for the coupling strength as function of the delay, respectively. The two curves intersect at τ = τmax , i.e. γmax (τmin ) = γmax (τmax ). Furthermore it holds that (a) γmin (τ ) → γ as τ → 0, (b) γmax (τ ) → ∞ as τ → 0. Remark 1. The data processing time of the coupling interface is responsible for the fact that timedelays τ 80[µs] are not included in the result in Fig. 10. Similar experiments are performed for the network configurations depicted in Fig. 11. The results are presented presented in Fig. 12. Remarkably, all results show a certain similarity with respect to the shape of the area for which the synchrony of the

0.5

0 0

0.5

1

1.5

2

2.5

3

Fig. 12. Experimental synchronization of the different time-delayed diffusively coupled networks Ge |e=1,2...,7 .

networks appear to be asymptotically stable. Next we will again investigate the effect of the network’s topology on its synchronous behavior. First, consider the three all-to-all configurations represented by G1 , G3 and G7 , respectively. One immediately notices the similar shapes of the stability regions. Moreover, each of these results appear to possess an equal value for the maximum allowable time-delay τmax in the network. This indicates that there is some kind of equivalence between those graphs. Next, we consider the ring configuration G5 and its identical configuration with an additional diagonal edge, see G6 . The stability regions are

June 3, 2010 12:3 00239

244


almost identical, i.e. for both graphs the shape of the stability diagram and the maximal allowable timedelay τmax are the same. Note that the results follow from experiments what might explain the small differences. It appears that the additional diagonal connection in G6 with respect to the ring configuration does not have any influence on the stability diagram. These results indicate that the maximal pathlength (distance) of the network plays an important role. However, the maximal pathlength is not the only thing that affect the stability region. For instance, both graphs G2 and G5 have the same a maximal pathlength but maximal allowable delay differs. The question is how the network topology influences the stability regions. We present some results in the next subsection.

5.2. Scaling of the γ and τ -axes So far we found that the maximum allowable timedelay in a network might be influenced by the maximal pathlength. Is it possible to scale the result of, for instance, two coupled systems to that of one of the other graphs? This would bring us to a conjecture similar to the Wu-Chua conjecture, see Conjecture 1, but then particularly for time-delays. In the limit τ → 0 we expect the same result as for the non-delayed coupling, i.e. there exists a threshold and if the coupling exceeds this threshold the neurons synchronize. These thresholds for the different graphs are related by the Wu-Chua conjecture. Let us therefore scale the results of all graphs to that of the two coupled systems in γ-direction using the Wu-Chua conjecture. The result is given in Fig. 13. Remarkably the graphs for the all-to-all connections seem to be almost identical after the scaling. Moreover, the lower bound on the coupling strength as function of τ seems to be about the same for all graphs up to the point where the maximal allowable time-delay for that graph is reached. The next question is whether we can predict the τmax for a certain graph based on information of the maximal time delay of two coupled systems. It appears that τmax scales with the ratio of the smallest nonzero eigenvalue λ2 and the largest eigenvalue λn of the

3

2.5

2

1.5

1

0.5

0 0

1

2

3

4

5

6

Fig. 13. Scaled experimental synchronization of the different time-delayed diffusively coupled networks Ge |e=1,2...,7 .

Table 3. The smallest nonzero eigenvalue λ2 and the largest eigenvalue λn of the coupling matrices Γ belonging to the graphs Gl |l=1,2,...,7 in Fig. 11. Ge

λ2

λn

λ2 /λn

τmax [ms]

G1 G2 G3

2γ γ 3γ 2 √ γ 2+ 2 2γ 2γ 4γ

2γ 3γ 3γ √ 2 + 2γ

1 1/3 1 2 √ (2 + 2)2 1/2 1/2 1

2.9 0.47 2.9

G4 G5 G6 G6

4γ 4γ 4γ

0.23 1.4 1.4 2.9

coupling matrix. When this ratio is equal to one, which is only the case in an all-to-al coupled network, the allowable time delay is maximal. If the ratio λλn2 decreases, which happens if the graph becomes more sparsely connected, the maximal allowable timedelay decreases, see Table 3. Moreover, if the product τmax λn is plotted versus λ2 , depicted in Fig. 14, it even appears that the product τmax λn is a proportional function of λ2 . Especially this result gives a strong claim for the existence of a scaling in time-delays between that of two time-delayed coupled systems and that of other time-delayed coupled network configurations. An exact scaling, however, is not found during this research.

June 3, 2010 12:3 00239


14

12

10

8

6

4

2

0 0.5

1

1.5

2

2.5

3

3.5

4

Fig. 14. Function of the smallest nonzero eigenvalue λ2 of the coupling matrix Γ against the product λn τmax .

6. Discussion We have presented results on synchronization in networks consisting of Hindmarsh-Rose neurons that interact via gap-junction (or electric synapses). In the first case we considered the case where timedelay was absent, i.e. an infinite propagation speed of the signals is assumed. We presented a framework which guarantees synchronization of the network whenever a certain threshold is exceeded. In practice this means that the coupling for a given network topology should be sufficiently strong. Since the Hindmarsh-Rose neurons satisfy the conditions of the framework we know that these neurons can be synchronized. In addition we discussed a conjecture which relates synchronization in one network to synchronization in a different network. The conjecture states that the system synchronize if and only if the smallest nonzero eigenvalue of the coupling matrix of network 1 equals the smallest nonzero eigenvalue of the coupling matrix of network 2. Since the smallest nonzero eigenvalue of the coupling matrix is related to the connectivity of the network it follows that a sparsely connected network requires stronger coupling to synchronize than a network with non-sparse connections. We demonstrated these results using an experimental setup consisting of up to sixteen electronic Hindmarsh-Rose neurons. The experimental results fully support the theoretical expectations. Next we included time-delay on the interaction which is induced by the finite propagation speed of membrane potentials through the axons

245

and dendrites. Unfortunately there are no rigorous mathematical results that guarantee synchronization in networks with time-delay. However, experimental results in networks consisting of up to four electronic neurons show that the synchronization is “robust” with respect to small time-delays, i.e. a sufficiently small time-delay will not destroy the synchronized state. The amount of time-delay that is allowed depends on the coupling strength. Moreover, there is a maximal time delay that is allowed and its value changes when the network topology changes. In networks with an all-to-all topology the maximal allowable time-delay seems to be identical. For any other network topology the allowable time delay will be lower than the time delay that is allowed in an all-to-all network. Just as in networks without time-delay there appear to apply some scaling laws in networks with time-delay. The minimal coupling strength required to synchronize can still be predicted by the Wu-Chua conjecture. Furthermore the maximal allowable time delay appear to decrease with λλn2 , where λ2 and λn represent the smallest nonzero eigenvalue and the maximal eigenvalue of the coupling matrix, respectively. As demonstrated there is a need to understand the effects of time-delays in networks, especially because time-delays will definitely emerge in networks of neurons due to the finite propagation speed of the transmitted signals. In the first we want to be able to guarantee that neurons can indeed synchronize even in the presence of time-delays. Existing methods to analyze synchrony in networks with delayed interaction can in general not be applied to networks of neurons24 or they give only very conservative results.22,23 Although it seems to be true that Hindmarsh-Rose neurons synchronize in presence of time-delay, it is not known if, for instance, HodgkinHuxley neurons will synchronize when time-delay is taken into account. (It is proved in (Ref. 25) that many neuronal models such as those of HodgkinHuxley and Morris-Lecar, fit into the theoretical framework presented in Sec. 4.1. Hence in absence of time-delays neurons will always synchronize provided that the coupling is sufficiently strong.) The next question is how the network topology affects the synchronization. For the coupling without delay we can apply the Wu-Chua conjecture. Thus we know how the topology changes the synchronous behavior since a different topology will result in a

June 3, 2010 12:3 00239

246


different smallest nonzero eigenvalue of the coupling matrix. Such results do not exist yet for networks with time-delay. However, improved knowledge of how the network topology in combination with timedelays affects synchronization would be extremely valuable for better understanding of the dynamics of networks of interconnected neurons. Since our experimental result imply the existence of such scaling laws in presence of time-delays it is definitely worth investigating. Appendix A Proof of Proposition 1. Following (Refs. 25 and 26). Consider the storage function

1 1 2 2 w2,i + , vi2 + µw1,i V (vi , w1,i , w2,i ) = 2 0.005 · 4 (A.1) with constant µ > 0. A straightforward computation shows that V˙ (vi , w1,i , w2,i ) ≤ vi ui − H(vi , w1,i , w2,i )

(A.2)

where H(vi , w1,i , w2,i ) = η1 vi4

(5 − 2µ)2 − 3+ vi2 + (8 − I)vi 4µ(1 − η2 )

µ 2 + µ η2 − w1,i 4(1 − η1 ) 1 2 4.472 w2,i + w2,i − 4 4

2 µ 2 w1,i + (1 − η1 ) vi + 2(1 − η1 )

2 5 − 2µ 2 vi , + (1 − η2 ) w1,i − 2µ(1 − η2 )

(A.3)

with 0 < η1 , η2 < 1 and 0 < µ < 4η2 (1 − η1 ). Clearly the function H(·, ·, ·) becomes positive for large enough col(vi , w1,i , w2,i . References 1. S. H. Strogatz and I. Stewart, Coupled oscillators and biological synchronization, Sci. Am. 269(6) (1993) 102–109. 2. C. S. Peskin, Mathematical aspects of heart physiology, Courant Institute of Mathematical Sciences, New York University (1975) 268–278.

3. A. Sherman, J. Rinzel and J. Keizer, Emergence of organized bursting in clusters of pancreatic beta-cells by channel sharing, Biophys. J. 54(3) (1988) 411– 425. 4. A. Pikovski, M. Rosenblum and J. Kurths, Synchronization — A Universal Concept in Nonlinear Sciences, Cambridge University Press (2007). 5. S. H. Strogatz, Sync: the emerging science of spontaneous order, Hyperion, New York, (2003). 6. C. M. Gray, Synchronous oscillations in neuronal systems: Mechanisms and functions, J. Comp. Neuroscience 1(1) (1994) 11–38. 7. W. Singer, Neuronal Synchrony: A Versitile Code for the Definition of Relations, Neuron 24 (1999) 49–65. 8. E. Mosekilde, Y. Maistrenko and D. Postnov, Chaotic Synchronization - Application to Living Systems, World Scientific Publishing (2002). 9. E. Niebur, H. Schuster, D. Kammen and C. Koch, Oscillator-phase coupling for different two dimensional network connectivities, Phys. Rev. A 44(10) (1991) 6895–6904. 10. S. Strogatz, Exploring complex networks, Nature 410 (2001) 268–276. 11. J. L Hindmarsh and R. M. Rose, A model for neuronal bursting using three coupled differential equations, Proc. R. Soc. Lond. B 221 (1984) 87–102. 12. M. V. L. Bennet and R. S. Zukin, Electrical coupling and neuronal synchronization in the mammalian brain, Neuron 41 (2004) 495–511. 13. C. Wu and L. Chua, On a conjecture regarding the synchronization in an array of linearly coupled dynamical systems, IEEE Trans. Circuits Syst. I 43(2) (1996) 161–165. 14. A. Pogromsky and H. Nijmeijer, Cooperative oscillatory behavior of mutually coupled dynamical systems, IEEE Trans. Circuits Syst. I 48(2) (2001) 152–162. 15. Young Jun Lee. Lee, Jihyun Lee, Y. B. Kim, A. Volkovskii, A Selverston, H. Abarbanel and M. Rabinovich, Low power real time electronic neuron VLSI design using subthreshold technique, Proc. of the 2004 International Symposium on Circuits and Systems 4 (2004) 744–747. 16. A. Pogromsky, Passivity based design of synchronizing systems, Int. J. Bifurcation and Chaos 8(2) (1998) 295–319. 17. J. C. Willems, Dissipative dynamical systems part I: General theory, Arch. Rational Mech. Anal. 45 (1972) 321–351. 18. B. P. Demidovich, Lecturs on Stability Theory, Nauka-Moscow, in Russian (1967). 19. A. Pavlov, A. Pogromsky, N. van de Wouw and H. Nijmeijer, Convergent dynamics, a tribute to Boris Pavlovich Demidovich, Systems Control Lett. 52 (2004) 257–261.

June 3, 2010 12:3 00239


20. L. Pecora and T. Carroll, Master Stability functions for synchronized coupled systems, Phys. Rev. Lett. 80(10) (1998) 2109–2112. 21. D. Watts and S. Strogatz, Collective dynamics of ‘small-world’ networks, Nature (London) 393 (2004) 440–442. 22. H. Huijberts, H. Nijmeijer and T. Oguchi, Anticipating synchronization of chaotic Lur’e systems, Chaos 17 (2007) 013117-1–013117-13. 23. T. Oguchi and H. Nijmeijer, Synchronization in Networks of Chaotic Systems with Time-delay Coupling, Chaos 18 (2008) 037108.

247

24. T. Oguchi, H. Nijmeijer and N. Tanaka, A synchronization condition for coupled nonlinear systems with time-delay — A circle criterion approach, Proc. 2nd IFAC Symposium on Chaos, London (2009). 25. E. Steur, I. Tyukin and H. Nijmeijer, Semi-passivity and synchronization of diffusively coupled neuronal oscillators, Physica D 238 (2009) 2119–2128. 26. W. T. Oud and I. Tyukin, Sufficient conditions for synchronization in an ensemble of hindmarsh and rose neurons: Passivity-based approach, 6th IFAC Symp. Nonlinear Control Systems, Stuttgart (2004).