754

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 57, NO. 3, MARCH 2012

Synchronization in Complex Networks With Stochastically Switching Coupling Structures Bo Liu, Wenlian Lu, and Tianping Chen, Senior Member, IEEE

Abstract—Synchronization in complex networks with time dependent coupling and stochastically switching coupling structure is discussed. A novel approach investigating synchronization based on the scramblingness property of the coupling matrix is proposed. Some sufficient condition for a network with general time-varying coupling structure to reach complete synchronization is provided. Based on the general theorem, networks with stochastically switching coupling structures is investigated. In particular, two kinds of stochastic switching coupling networks are addressed: (a) independent and identically distributed switching processes and (b) Markov jump processes. In both cases, some sufficient condition for almost sure synchronization of the networks is given. Also, numerical simulations are provided to illustrate the theoretical results. Index Terms—Coupled system, stochastic systems, switched systems, synchronization, time-varying.

I. INTRODUCTION Today, the study of synchronization in complex dynamical systems has become a subject of great interest due to its applications and potential applications in a variety of fields, such as communication [1], seismology [2], and neural networks [3]. Till now, many works are available engaging in the study of synchronization of complex networks. For example, in the pioneering work [4], Master Stability Function (MSF) method to study the local synchronization of coupled chaotic systems was proposed. In [5]–[7], the distance to the synchronization manifold was defined, and some sufficient conditions for an array of linearly coupled systems to synchronize were proposed. Most of the works on synchronization are focused on static networks, i.e., the coupling structure and coupling strength is constant in time. In such case, the criteria for synchronization have been well-established by analyzing the eigen-structure of the coupling matrix. Besides, there are also some papers concerning synchronization in dynamic networks, i.e., the network coupling structure and coupling strength is dynamically changing along with time. Here, we list some (not all) relating papers [9]–[13], [15], [20], etc. In [9], the authors studied global synchronization in a blinking network model. They used the connection graph stability method developed in [8]. In [20], the authors investigated synchronization in networks of coupled Kuramoto oscillators with switching topologies and time delays. This model also can be viewed as nonlinear consensus (see [25]). In [12], sufficient conditions for fast switching synchronization in networks with timevarying topologies were given. In [15], the authors studied synchronization in networks with random switching topologies and gave sufficient conditions for almost sure local synchronization. Both [12] and [15] indicate that under the assumption of fast switching, the synchronization of the time-varying network can be deduced from their timeaverage system. Manuscript received June 09, 2010; revised January 24, 2011; accepted July 16, 2011. Date of publication September 01, 2011; date of current version February 29, 2012. This work was supported by the National Science Foundation of China under Grant 60974015, the Graduate Innovation Foundation of Fudan University under Grant EYH1411040. Recommended by Associate Editor M. Egerstedt. The authors are with the School of Mathematical Sciences, Fudan University, Shanghai 200433, China (e-mail: [email protected]; [email protected]; [email protected]). Digital Object Identifier 10.1109/TAC.2011.2166665

Another topic closely relating to synchronization problem is consensus problem in networks of multiagents. [16]–[19], [21]–[30] are a few among them. Though consensus problem is a special case of synchronization problem, some efficient approach used in the consensus problem can still be applied to investigation of synchronization problem. It is also known that if the networks are time-varying or with switching topologies, the synchronization or consensus becomes complicated [10], [14], [28]. Because it is difficult to construct a Lyapunov function when the coupling matrices are time varying, except the switching occurs among several strongly connected and balanced graphs. In such case, it is proved that for arbitrary switching, average consensus can be reached exponentially (see [16]). Instead, if the graph is not node balanced, it is difficult to find a common Lyapunov function so that this approach can not apply. Another efficient approach comes from the theory of nonhomogenous Markov chains by reducing the convergence of consensus algorithm to the ergodicity of infinite products of stochastic matrices. This method is widely used in [21]–[23], [27] and others. It was based on the work of Hajnal back to the 1950’s [31]. Hajnal investigated the weak ergodicity of non-homogenous Markov chains and proposed scrambling matrix, which plays an important role in the convergence of products of stochastic matrices. Similar method has also been used to study consensus problem in continuous time networks in [17], [18], [28], etc. This method was also used to discuss synchronization in [13]. It is natural to ask if this method can be further extended to more general cases in synchronization analysis. This is the aim of this technical note. In the following, we first address global synchronization in networks with a general time-varying topology and sufficient condition for global synchronization is given. To this purpose, we extend the concept of Hajnal’s scrambling property from stochastic matrices to matrices with nonnegative off-diagonal entries. Then we will turn to stochastic switching networks. Particularly, we will study synchronization for networks with two kinds of stochastically switching topologies. That is: (a) the switching sequence are independent and identically distributed; (b) the switching sequence forms a Markov chain. In both cases, we give sufficient conditions for the network to synchronize almost surely. In previous works, the authors considered either special node dynamics such as Kuramoto model in [20], or linear dynamics such as in [13], or local synchronization as in [15]. Instead, in this note, we consider global synchronization for continuous-time networks with general nonlinear node dynamics and general time varying topologies. In case the stochastic switching topologies, we don’t require the network to switch fast enough, while this requirement is assumed in [9], [12]. We also point out that if there is a nonzero probability of a scrambling coupling matrix, then the network will synchronize almost surely if the coupling strength is strong enough. The rest of the technical note is organized as follows. In Section II, we study networks with general time-varying topologies and provide a sufficient condition for such networks to achieve synchronization. In Section III, we study stochastic network and give sufficient conditions for almost sure synchronization. An example with numerical simulations are provided in Section IV, and the technical note is concluded in Section V. II. GENERAL THEORY In this section, we discuss synchronization in networks with general time dependent coupling.

0018-9286/$26.00 © 2011 IEEE

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 57, NO. 3, MARCH 2012

Consider the following coupled dynamic system:

dxi (t) dt

=

m

f (xi (t); t) + j

=1

aij (t)0xj (t);

755

FVij (t) =

n

2 + 7!

aij (t) + aji (t) + k

6=i;j

min

+ k (t =

1

k t!+1

0

j

=1

Then by the condition lim

!+1

t

FV

j

i

0 [ FVi(s)j (s) (s)

have lim V (t) = 0, and this can complete the proof. Now we are to t!+1 prove (2). At any given time t0

which means

xki((tt )) (t0 ) =

max fxk (t ) g =1 ;...;m j (t ) xk(t ) (t0 ) = min fxki(t ) g i=1;...;m i

i

and for any j = 6 i(t0 ), j = 6 j (t0 ), we have

xki((tt )) (t0 ) xjk(t ) (t0 ) xkj((tt )) (t0 ):

=1 j 6=( ); j (t ) j i t

fai(t )j ; aj (t )j g

min

0 xjk(t ) (t0 ) 0 xkj((tt )) (t0 )

j

=1

fai(t )j ; aj(t )j g

min

6=i(t );j (t )

ai(t )j (t ) + aj (t )i(t )

xki((tt )) (t0 ) 0 xkj((tt )) (t0 )

L 0 FVi(t )j (t ) (t0 ) xik((tt )) (t0 ) 0 xkj((tt )) (t0 )

which means + V (t0 ) L 0 FV D i(t ) j (t ) (t0 ) V (t0 ):

(4)

t, synchronization analysis can be reduced to the analysis of the eigen-

0 L] ds = +1, we

V (t0 ) = xki((tt )) (t0 ) 0 xkj((tt )) (t0 )

m

)

Remark 1: In case that the coupling matrix is independent of time

(s)ds :

t

=1

+

1t!0

0 V (t) V (0)e

j

m

synchronization: < = minf i g.

+ V (t) = lim sup ((V (t + 1t) 0 V (t))=1t). If this is done, where D

Lt

=1

ai(t )j xjk(t ) (t0 ) 0 xik((tt )) (t0 )

L xki((tt )) (t0 ) 0 xkj((tt )) (t0 ) 0 k (t )

!+1 0 [ FVi(s)j (s) (s) 0

(2)

j

aj (t )j xjk(t ) (t0 )

m

)

xjk(t ) (t0 ) 0 xki((tt )) (t0 )

2

t

+ V (t) [L 0 FV (t)]V (t) D i(t)j (t)

=1

m

aj (t )j xjk(t ) (t0 ) 0 xkj((tt )) (t0 )

+ k (t

t

k

j

ai(t )j xjk(t ) (t0 ) 0

L xki((tt )) (t0 ) 0 xkj((tt )) (t0 ) 0 k(t ) ai(t )j(t ) + aj(t )i(t ) 2 xik((tt )) (t0 ) 0 xkj((tt )) (t0 )

Proof: Let y (t) = [y1 (t); . . . ; yn (t)] 2 n be such that yk (t) = i i maxi fxk (t)g 0 mini fxk (t)g. Let V (t) = ky (t)k. We will prove that

we have

m

0

faik (t); ajk (t)g:

= + , the system (1) can achieve x (t) xj (t) = 0 holds for all i; j , where 0 i

m

)

fk(t ) (xi(t ) (t0 ); t0 ) 0 fk(t ) (xj (t ) (t0 ); t0 )

+ k (t

i;j

ds lim

= 0,

t=t i(t ) = fk (t ) (x (t0 ); t0 ) 0 fk(t ) (xj(t ) (t0 ); t0 )

n

fi(t); j (t); k(t)g. In this case, we just arbitrarily pick one). Then we L]

=1 aj (t )j

m j

dt

Let xi (t) be the solution of (1) from the initial value xi (0), and j (t); k(t) be the index satisfying xki((tt)) (t) 0 xkj((tt)) (t) = let i(t); i j max kx (t) 0 x (t)k ( There may be more than one choice of can have: Theorem 1: Under Assumption 1, if lim

=

xki((tt )) (t) 0 xkj((tt )) (t)

d

i = 1; 2; . . . ; m (1)

where x 2 is the state vector, and f : is a continuous map which represents the dynamics of an uncoupled node, > 0 is the coupling strength, A(t) = [aij (t)] is the weighted outer coupling matrix that has nonnegative off diagonal entries and zero row sum for each t, and aij (t) is piecewise continuous and integrable on any finite interval of + so that standard solution of (1) exists and is unique. A(t) is also known as the negative of the Laplacian matrix of the underlying graph. 0 = diag[ 1 ; 2 ; . . . ; n ] is the weighted inner coupling matrix which is positive definite. Throughout this technical note, for x = [x1 ; x2 ; . . . ; xn ]> 2 n , kxk = max jxi j. We always make the following assumption on f : i Assumption 1: f (x; t) is Lipschitz continuous in x, with the Lipschitz constant being L > 0, i.e., kf (x; t) 0 f (y; t)k Lkx 0 y k. Let A(t) = [aij (t)] be matrices with nonnegative off-diagonal entries, for i; j 2 f1; 2; . . . ; mg, i 6= j , define a function n

i

=1 ai(t )j

m j

Differentiating and noting that we have

(3)

vectors structure of the coupling matrix. Based on the left eigenvector T = [1 ; . . . ; m ] corresponding to the eigenvalue 0, some effective synchronization criteria are given (for example, see [6], [7], where it x (t) ! 0). was proved that under some assumptions, xi (t) 0 n i=1 i i However, if the coupling matrix is time dependent, the eigenvector (t) = [1 (t); . . . ; m (t)]T corresponding to the eigenvalue 0 is also time dependent, except the coupling matrix is node-balanced. In this case, the left eigenvector corresponding to the eigenvalue 0 is (t) = T [1; . . . ; 1] , which is time independent. Therefore, all the derivation for constant coupling matrix can be used without any modifications. In this note, different from [9]–[12], [15], we propose another approach in discussing of synchronization for time-varying complex networks. Though the condition required in Theorem 1 seems difficult to verify, it is still effective for the networks with stochastically switching coupling structures. For details, see the following sections. Remark 2: When f 0, synchronization becomes consensus problem. In this case, a useful concept “ graph” was introduced in [18]. It was proved that if the connection graph satisfies that all aij (t) t+T are bounded and the graph associated to (1=T ) t A(s)ds > for

756

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 57, NO. 3, MARCH 2012

some > 0, T > 0 and all t 0 has a spanning tree with a fixed root vertex, then consensus can be realized. Based on Theorem 1, we have Corollary 1: Under Assumption 1, if

First, it is clear that when t 2 [tk01 ; tk ), FVi(t)j (t) (t) RV(Ak ). Let Kt = maxfk; tk tg, from Theorem 7.3 of [32], we can have

+

t T

1 T

min aij (t)+ aji (t)+ i;j

t

6=

lim

!1

t

minfaik (t); ajk (t)g dt >

k i;j

Then by the strong law of large numbers for i.i.d. sequence, we have

for some > 0, T > 0 and all t 0, particularly, if the coupling graph is of scrambling property for each t, i.e., for any indices i; j , i 6= j , either one of the following conditions satisfies: (1). aij (t)+ aji (t) > ; (2). There is an index k 2 f1; . . . ; ngnfi; j g such that aik (t) > > 0 and ajk (t) > > 0, then the system (1) can achieve synchronization when the coupling strength is large enough.

t

lim

!+1

t

0 FVi(s)j (s) (s)ds 0 L t

K

1

K !lim +1 t

t

Kt

t

k

In this section, we will investigate synchronization in networks with stochastically switching topology. l n ]i;j =1 2 Rn2n ; l = Let fSG ; Pg, where SG = fGl : jGl = [gij 1; . . . ; N g, P = fpl : pl > 0; l = 1; . . . ; N; N l=1 pl = 1g, be a finite sttes probability space. Furthermore, we assume that each Gl is a matrix with zero row sum and nonnegative off diagonal entries. For any A, define RV(A) = minfaij + aji + k6=i;j minfaik ; ajk gg. Then, RV (G) dei;j

l l fined by RV(Gl ) = minfgij + gji + i;j

random variable on SG . Consider

m

j

=1

k

l l 6= minfgik ; gjk gg is a

k i;j

j

aij 0x (t); t

2 [t 01 ; t k

k

);

i = 1; . . . ; m

(5)

where Ak = [akij ] 2 SG . Denote 1tk = tk 0 tk01 . In the following, we assume: (I). the sequence f1tk g is independent and identically distributed random vari~ and E f1t2k g < 1, i.e., ftn g forms a ables such that E f1tk g = 1 renewal process; (II). the switching time and topology switching are independent; (III). with probability 1, there are only finite times of switching occurs on each finite time interval, which excludes the possibility of infinitely fast switching and ensures the existence and uniqueness of the standard solution of (5).

In this subsection, we assume that fAk g is an i.i.d. sequence with probability distribution

PfAk = Gl(k) g = pl(k) ; l = 1; 2; . . . ; N:

t

t t

RV(Ak )ds

0L

t

k

k

k

k

holds almost surely. This completes the proof. B. Markov Switching Process In this subsection, we assume that the sequence fAk g is a homo= [pij ], where geneous Markov chain with the transition Matrix PfAk+1 = Gj jAk = Gi g = pij . And we assume that is irreducible, aperiodic, and has a unique stationary distribution . Similarly, we can have the following: Theorem 3: Under Assumption 1, if the expectation of RV(Gi ) with respect to the stationary distribution satisfies E fRV(Gi )g > L= , where = minf i g, the system (5) will achieve synchronization ali most surely. Proof: Similar to the proof of Theorem 2, we show t

lim

FVi(s)j (s) (s)

!+1 0

t

ds = +1:

0L

Using the fact that FVi(t)j (t) (t) RV(Ak ) when t 2 [tk01 ; tk ), and by the strong law of large numbers for Markov chains, we have

lim

!+1

t

A. I.I.D. Switching Process

=1

RV(Ak )ds +

K

t

III. MAIN RESULTS AND PROOF

t

K 1 !lim +1 t K =1 RV(A )1t 0 L 1 = EfRV(G)gEf1t g 0 L ~ 1 = EfRVg 0 L > 0 t

dxi (t) = f (xi (t); t) + dt

1 Kt = : ~ t 1

t 0 FVi(s)j (s) (s)ds 0 L

t

1

K !lim +1 t

t

Kt

t

K !lim +1 t

t

t

1

1 Kt

K

k

t

=1

t

K

k

=1

t t

RV(Ak )ds +

RV(Ak )ds

t t

RV(Ak )ds

0L

0L

E fRVgE1tk 0 L ~ !+1 1

= lim t

l(k)

2 f1; . . . ; N g depends on k. Then we have

Theorem 2: Under Assumption 1, if the expectation of N EfRV(G)g satisfies EfRV(G)g = > L= , l=1 RV(Gl )pl where = minf i g, the system (5) will reach synchronization almost i surely. Proof: From Theorem 1, it suffices to prove

P

!lim +1 0

t

t

FVi(s)j (s) (s)

0L

ds = +1 = 1:

= E fRVg 0 L > 0

holds almost surely. This completes the proof. Remark 3: It is necessary to compare the results on synchronization for switching systems obtained in this note and those reported in literature such as [9], [12], [15] and others. Briefly, in [9], [12], [15], some additional requirements on the switching speed of the network topology, the so-called ”fast-switching,” are imposed. And in [9], [15], the switching time intervals are assumed equally spaced. In such

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 57, NO. 3, MARCH 2012

757

fast enough, the switching system is ”near” to the average system in some sense. Or we can say it can approximately be the average system. Therefore, the synchronization can be reached by analyzing the average system. In this technical note, we do not require fast-switching. And compared to [9], which considers a special small- world networks generated from a regular 2 lattice, we consider a much general network structure. Yet, in this note, the coupling graph is required to have scrambling-like structure. As for [12], [15], the authors considered local stability of the synchronization manifold by linearization technique. Instead, we consider global synchronization here.

K

IV. NUMERICAL SIMULATIONS In this section, we will provide a numerical example to illustrate the theoretical results. In the following, we consider:

dx (t) = f (x (t); t) + dt i

5

i

j

a x (t); =1 t 2 [t 01 ;t ); i = 1; . . . ; 5: k ij

j

k

I

That is 0 = 3 , and assumed as

k

(6)

= 1. The dynamics of uncoupled systems are

f (x;t) = [sin(t)x; cos(t)x; sin(t + 1)x)]>

(7)

or Chua’s circuit

p[0x1 + x2 + g(x1 )] f (x; t) = x1 0 x2 + x3 (8) 0qx2 where g (x1 ) = m0 x1 + (1=2)(m1 0 m0 )(jx1 + 1j 0 jx1 0 1j). m0 = 00:68, m1 = 01:27, p = 9:0, q = 12:87. Let S = fG1 ; G2 ; G3 g, where 3 matrices 02:2889 0:8464 0:9573 0:2553 0:2299 0:6423 02:6972 0:6203 0:8586 0:5761 G1 = 0:2213 0:2789 02:2218 0:9111 0:8106 0:8371 0:7466 0:1726 02:1601 0:4038 0:9711 0:2369 0:0903 0:7252 02:0235 01:4357 0:5566 0:3178 0:2699 0:2915 0:3209 02:1481 0:4522 0:5246 0:8504 G2 = 0:5114 0:8300 03:2257 0:9727 0:9116 0:0606 0:8588 0:1099 01:6685 0:6393 0:7257 0:7890 0:1097 0:3119 01:9363 01:5879 0:5846 0:3934 0:1338 0:4761 0:8383 03:2444 0:8266 0:6715 0:9081 G3 = 0:5847 0:8277 02:5356 0:5710 0:5522 0:9481 0:1910 0:2076 01:3796 0:0329 0:0610 0:4425 0:3181 0:1477 00:9693 are picked randomly. Simple calculations show that RV(G1 ) = 1:7835, RV(G2 ) = 1:2884, RV(G3 ) = 0:6402. In the following simulations, we assume that each 1t be uniformly distributed on [0; 1]. The synchronization error is defined as err(t) = 5 1 2 =2 kx 0 x k . And the initial value are chosen randomly. G

k

i

i

A. I.I.D. Case

Fig. 1. Dynamics of x , x , x and synchronization error for the system (6) with f(x; t) being given by (7) for i.i.d. switching topology.

case, they proved that the switching system can reach synchronization (with probability 1) if the corresponding average system (or expectation system) can reach synchronization. When the switching is

In this subsection, let fA g be i.i.d. with PfA = G1 g = PfA = G2 g = PfA = G3 g = 1=3. Simulation 1: Assume that the dynamics of the uncoupled node f (x; t) is governed by (7) and = 1. Then L = 1, and EfRVg = 1:2374 > L. By Theorem 2, the system (7) can reach synchronization almost surely. The results of the simulation are provided in Fig. 1, which shows the dynamical behavior of x1 (t), x2 (t), x3 (t), i = 1, 2, k

k

k

k

i

i

i

758

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 57, NO. 3, MARCH 2012

Fig. 2. Dynamics of x (t), x (t), x and synchronization error for the system (6) with f(x; t) being Chua’s circuit (8) for i.i.d. switching topology.

Fig. 3. Dynamics of x , x , x and synchronization error for the system (6) with f(x; t) being given by (7) for Markovian switching topology.

3, 4, 5, and err(t), respectively. It can be seen that the system actually achieves synchronization.

Simulation 2: Now, we assume that the dynamics f (x; t) of the uncoupled node is governed by Chua’s circuit (8) and = 10:5. Then

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 57, NO. 3, MARCH 2012

759

B. Markov Chains In this subsection, we will consider the case that fAk g forms a Markov chain with the transition matrix

0:5311 0:3195 0:1494 = 0:5422 0:3503 0:1075 such that the unique stationary 0:5746 0:1244 0:3011 distribution = [0:5414; 0:2972; 0:1614]. Simulation 3: Let f (x; t) be given by (7), and = 1, then E fRVg = 1:4518 > 1 = L= . Therefore, By Theorem 3, the

system (7) can reach synchronization almost surely. The simulation results are presented in Fig. 3. Simulation 4: Let f (x; t) be given by Chua’s circuits (8), and = 9, then E fRVg = 9 2 1:4518 = 13:0662 > L= . By Theorem 3, the system (7) can reach synchronization almost surely. The simulation results are presented in Fig. 4. Remark 4: If we consider the following system:

dxi (t) = f (xi (t); t) + m ak 0xj (t); ij dt j =1 t 2 [tk01 ; tk ); i = 1; . . . ; m (9) where is the coupling strength and RV(G) = RV(G). By Theorem 2 and Theorem 3, we can conclude that if f satisfies the Lipschitz con-

dition, the switching topology is i.i.d. or Markovian, and the expectation EfRVg > 0 (or E fRVg > 0), then the system (9) can reach synchronization almost surely, if is large enough. Remark 5: It should be pointed out that the theoretical is quite conservative. In fact, lots of numerical experiments indicate that the system can reach synchronization even the coupling strength is much smaller than the theoretical value. For example, in simulation 4, even pick = 1, synchronization can be reached, too. We also can use adaptive algorithm to search small coupling strength in practice. V. CONCLUSION In this note, we discuss synchronization in networks with time varying coupling, i.e., the coupling structure as well as coupling weights are time dependent. Motivated by the scrambling matrix used in the study of consensus problems, we propose a novel approach without using eigen-structure of the coupling matrix. Based on the discussion of complete synchronization in networks with general time-varying coupling structure, we discuss networks with stochastically switching coupling structures. Particularly, we discuss two kinds of stochastic networks: a) independent and identically distributed switching processes and b) Markov jump processes. In both cases, we give a sufficient condition for almost sure synchronization of the networks. Also, numerical simulations are provided to illustrate the theoretical results.

REFERENCES

Fig. 4. Dynamics of x (t), x (t), x and synchronization error for the system (6) with f(x; t) being Chua’s circuit (8) for Markovian switching topology.

L = 12:87 and EfRV g = 10:5 2 1:2374 = 12:9927 > L. By The-

orem 2, the system (7) can reach synchronization almost surely. The simulation results are provided in Fig. 2.

[1] G. D. Van Wiggeren and P. Roy, “Communication with chaotic laser,” Science, vol. 279, no. 20, pp. 1198–1200, 1998. [2] M. de S. Vieira, “Chaos and synchronized chaos in an earthquake model,” Phys. Rev. Lett., vol. 82, no. 1, pp. 201–204, 1999. [3] F. C. Hoppensteadt and E. M. Izhikevich, “Pattern recognition via synchronization in phase-locked loop neural networks,” IEEE Trans. Neural Netw., vol. 11, no. 3, pp. 734–738, May 2000. [4] L. M. Pecora and T. L. Carroll, “Master stability function for synchronized coupled systems,” Phys. Rev. Lett., vol. 80, pp. 2109–2112, 1998. [5] C. W. Wu and L. O. Chua, “Synchronization in an array of linearly coupled dynamical systems,” IEEE Trans. Circuits Syst. I, vol. 42, no. 8, pp. 430–447, Aug. 1995. [6] W. L. Lu and T. P. Chen, “Synchronization of coupled connected neural networks with delays,” IEEE Trans. Circuits Syst. I, vol. 51, no. 12, pp. 2491–2503, Dec. 2004. [7] W. L. Lu and T. P. Chen, “New approach to synchronization analysis of linearly coupled ordinary differential systems,” Physica D, vol. 213, pp. 214–230, 2006.

760

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 57, NO. 3, MARCH 2012

[8] V. N. Belykh, I. V. Belykh, and M. Hasler, “Connection graph stability method for synchronized coupled chaotic systems,” Physica D, vol. 195, pp. 159–187, 2004. [9] I. V. Belykh, V. N. Belykh, and M. Hasler, “Blinking model and synchronization in small-world networks with a time-varying coupling,” Physca D, vol. 195, pp. 188–206, 2004. [10] W. Wu and T. P. Chen, “Global synchronization criteria of linearly coupled neural network systems with time-varying coupling,” IEEE Trans. Neural Netw., vol. 19, no. 2, pp. 319–332, Feb. 2008. [11] J. D. Skufca and E. Bollt, “Communication and synchronization in disconnected networks with dynamic topology: Moving neighbourhood netoworks,” Math. Biosci. Eng., vol. 1, no. 2, pp. 347–359, 2004. [12] D. J. Stilwell, E. M. Bollt, and D. G. Roberson, “Sufficient conditions for fast switching synchronization in time-varying network topologies,” Siam J. Appl. Dyn. Syst., vol. 5, no. 1, pp. 140–156, 2006. [13] C. W. Wu, “Synchronization and convergence of linear dynamics in random directed networks,” IEEE Trans. Autom. Control, vol. 51, no. 7, pp. 1207–1210, Jul. 2006. [14] W. L. Lu, F. M. Atay, and J. Jost, “Synchronization of discrete-time dynamical networks with time-varying couplings,” SIAM J. Math. Anal., vol. 39, no. 4, pp. 1231–1259, 2007. [15] M. Porfiri, D. J. Stilwell, and E. M. Bollt, “Synchronization in random weighted directed networks,” IEEE Trans. Circuits Syst. I, vol. 55, no. 10, pp. 3170–3177, Nov. 2008. [16] R. Olfati-Saber and R. M. Murray, “Consensus problems in networks of agents with switching topology and time-delays,” IEEE Trans. Autom. Control, vol. 49, no. 9, pp. 1520–1533, Sep. 2004. [17] Y. Hatano and M. Mesbahi, “Agreement over random networks,” IEEE Trans. Autom. Control, vol. 50, no. 11, pp. 1867–1872, Nov. 2005. [18] L. Moreau, “Stability of continuous-time distributed consensus algorithms,” in Proc. 43rd IEEE Conf. Decision Control, Atlantis, Paradise Island, Bahamas, Dec. 14–17, 2004, pp. 3998–4003. [19] L. Moreau, “Stability of multiagent systems with time-dependent communication links,” IEEE Trans. Autom. Control, vol. 50, no. 2, pp. 169–182, Feb. 2005. [20] A. Papachristodoulou and A. Jadbabaie, “Synchronization in oscillator networks: Switching topologies and non-homogenous delays,” in Proc. 44th IEEE Conf. Decision Control, Eur. Conf., Seville, Spain, Dec. 12–15, 2005, pp. 5692–5697. [21] A. Tahbaz-Salehi and A. Jadbabaie, “Necessary and sufficient conditions for consensus over random independent and identically distributed switching graphs,” in Proc. 46th IEEE Conf. Decision Control, New Orleans, LA, Dec. 12–14, 2007, pp. 4209–4214. [22] A. Tahbaz-Salehi and A. Jadbabaie, “A necessary and sufficient condition for consensus over random networks,” IEEE Trans. Autom. Control, vol. 53, no. 3, pp. 791–795, Mar. 2008. [23] A. Tahbaz-Salehi and A. Jadbabaie, “Consensus over ergodic stationary graph processes,” IEEE Trans. Autom. Control, vol. 55, no. 1, pp. 225–230, Jan. 2010. [24] A. Papachirstodoulou, A. Jadbabaie, and U. Münz, “Effects of delay in multi-agent consensus and oscillator synchronization,” IEEE Trans. Autom. Control, vol. 55, no. 6, pp. 1471–1477, Jun. 2010. [25] X. W. Liu, W. L. Lu, T. P. Chen, and W. Lu, “Consensus problem in directed networks of multi-agents via nonlinear protocols,” Phys. Lett. A, vol. 373, pp. 3122–3127, 2009. [26] X. W. Liu, W. L. Lu, and T. P. Chen, “Consensus of multi-agent systems with unbounded time-varying delays,” IEEE Trans. Autom. Control, vol. 55, no. 10, pp. 2396–2401, Oct. 2010. [27] S. Boyd, A. Ghosh, and B. Prabhakar, “Randomized gossip algorithms,” IEEE Trans. Inform. Theory, vol. 52, no. 6, pp. 2508–2530, Sep. 2006. [28] B. Liu and T. P. Chen, “Cosensus in networks of multiagents with cooperation and competition via stochastically switching topologies,” IEEE Trans. Neural Netw., vol. 19, no. 11, pp. 1967–1973, Nov. 2008. [29] B. Liu, W. L. Lu, and T. P. Chen, “Reaching Lp consensus in a network of multiagents with stochastically switching topologies,” in Proc. Joint 48th IEEE Conf. Decision Control 28th Chinese Control Conf., Shanghai, China, Dec. 16–18, 2009, pp. 2670–2675. [30] B. Liu, W. L. Lu, and T. P. Chen, “Consensus in networks of multiagents with switching topologies modeled as adapted stochastic processes,” SIAM J. Control Optim., vol. 49, no. 1, pp. 227–253, 2011. [31] J. Hajnal, “Weak ergodicity in non-homogenous Markov chains,” in Proc. Cambridge Philosophical Soc., 1958, vol. 54, pp. 233–246. [32] R. Durrett, Probability: Theory and Examples. Belmont, CA: Duxbury Press, 2005.

A Riccati Based Interior Point Algorithm for the Computation in Constrained Stochastic MPC Minyong Shin and James A. Primbs

Abstract—We propose a fast algorithm for the linear-quadratic control problem with probabilistic constraints that is repeatedly solved in stochastic model predictive control. Under the assumption of affine state feedback and Gaussian noise, the finite horizon control problem is converted to an equivalent deterministic problem using the mean and covariance matrix as the state. A line search interior point method is proposed to solve this optimization problem, where the step direction can be quickly computed via a Riccati difference equation. Numerical examples show that this algorithm has linear complexity in the horizon length. Index Terms—Interior point methods (IPMs), model predictive control (MPC).

I. INTRODUCTION In this technical note, we propose a fast but suboptimal algorithm for the finite horizon control of a discrete-time linear dynamic system with additive Gaussian noise under a quadratic objective and linear probabilistic constraints. While control problems of this type are of interest in their own right, their use in stochastic model predictive control (MPC) schemes accentuates the importance of fast algorithms that can be used to produce real-time solutions. Stochastic MPC is an important research area in the control community (see for example [3], [4], [10], [16], [19], [20]), and is the primary motivation for this work. Significant progress has been made in the development of Stochastic MPC methods. In particular, issues of stability and recursive feasibility have garnered attention [13], [14], and important advances have been made in the numerical on-line computations. Specifically, the numerical solution of finite horizon stochastic LQ problems with probabilistic chance constraints has received much attention, especially in the direction of developing a convex formulation. In a series of papers, van Hessem and Bosgra showed that under affine feedback assumptions it could be solved as a convex optimization problem by using a Youla parameterization [23] or “innovation feedback” [24]. Löfberg [12] and Goulart et al. [6] used the related idea of disturbance feedback to provide a convex formulation under bounded disturbances. While these convex formulations allow SDP solvers to be used, a drawback is that computation time grows quickly with problem size and horizon length. To counter this, Bertsimas and Brown [1] and Bertsimas and Sim [2] have provided tractable convex formulations in an open loop setting, and Oldewurtel et al. [18] followed up on this work with a linear programming approximation under affine feedback. The purpose of this technical note is to present a fast algorithm for the finite horizon linear-quadratic-Gaussian problem under affine state Manuscript received September 11, 2010; revised February 15, 2011; accepted June 13, 2011. Date of publication September 15, 2011; date of current version February 29, 2012. This technical note was presented in part at the American Control Conference, 2010. Recommended by Associate Editor C. Szepesvari. M. Shin is with the Department of Aeronautics and Astronautics, Stanford University, Stanford, CA 94305 USA (e-mail: [email protected]). J. A. Primbs is with the Department of Management Science and Engineering, Stanford University, Stanford, CA 94305 USA (e-mail: [email protected]). Color versions of one or more of the figures in this technical note are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TAC.2011.2168069

0018-9286/$26.00 © 2011 IEEE

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 57, NO. 3, MARCH 2012

Synchronization in Complex Networks With Stochastically Switching Coupling Structures Bo Liu, Wenlian Lu, and Tianping Chen, Senior Member, IEEE

Abstract—Synchronization in complex networks with time dependent coupling and stochastically switching coupling structure is discussed. A novel approach investigating synchronization based on the scramblingness property of the coupling matrix is proposed. Some sufficient condition for a network with general time-varying coupling structure to reach complete synchronization is provided. Based on the general theorem, networks with stochastically switching coupling structures is investigated. In particular, two kinds of stochastic switching coupling networks are addressed: (a) independent and identically distributed switching processes and (b) Markov jump processes. In both cases, some sufficient condition for almost sure synchronization of the networks is given. Also, numerical simulations are provided to illustrate the theoretical results. Index Terms—Coupled system, stochastic systems, switched systems, synchronization, time-varying.

I. INTRODUCTION Today, the study of synchronization in complex dynamical systems has become a subject of great interest due to its applications and potential applications in a variety of fields, such as communication [1], seismology [2], and neural networks [3]. Till now, many works are available engaging in the study of synchronization of complex networks. For example, in the pioneering work [4], Master Stability Function (MSF) method to study the local synchronization of coupled chaotic systems was proposed. In [5]–[7], the distance to the synchronization manifold was defined, and some sufficient conditions for an array of linearly coupled systems to synchronize were proposed. Most of the works on synchronization are focused on static networks, i.e., the coupling structure and coupling strength is constant in time. In such case, the criteria for synchronization have been well-established by analyzing the eigen-structure of the coupling matrix. Besides, there are also some papers concerning synchronization in dynamic networks, i.e., the network coupling structure and coupling strength is dynamically changing along with time. Here, we list some (not all) relating papers [9]–[13], [15], [20], etc. In [9], the authors studied global synchronization in a blinking network model. They used the connection graph stability method developed in [8]. In [20], the authors investigated synchronization in networks of coupled Kuramoto oscillators with switching topologies and time delays. This model also can be viewed as nonlinear consensus (see [25]). In [12], sufficient conditions for fast switching synchronization in networks with timevarying topologies were given. In [15], the authors studied synchronization in networks with random switching topologies and gave sufficient conditions for almost sure local synchronization. Both [12] and [15] indicate that under the assumption of fast switching, the synchronization of the time-varying network can be deduced from their timeaverage system. Manuscript received June 09, 2010; revised January 24, 2011; accepted July 16, 2011. Date of publication September 01, 2011; date of current version February 29, 2012. This work was supported by the National Science Foundation of China under Grant 60974015, the Graduate Innovation Foundation of Fudan University under Grant EYH1411040. Recommended by Associate Editor M. Egerstedt. The authors are with the School of Mathematical Sciences, Fudan University, Shanghai 200433, China (e-mail: [email protected]; [email protected]; [email protected]). Digital Object Identifier 10.1109/TAC.2011.2166665

Another topic closely relating to synchronization problem is consensus problem in networks of multiagents. [16]–[19], [21]–[30] are a few among them. Though consensus problem is a special case of synchronization problem, some efficient approach used in the consensus problem can still be applied to investigation of synchronization problem. It is also known that if the networks are time-varying or with switching topologies, the synchronization or consensus becomes complicated [10], [14], [28]. Because it is difficult to construct a Lyapunov function when the coupling matrices are time varying, except the switching occurs among several strongly connected and balanced graphs. In such case, it is proved that for arbitrary switching, average consensus can be reached exponentially (see [16]). Instead, if the graph is not node balanced, it is difficult to find a common Lyapunov function so that this approach can not apply. Another efficient approach comes from the theory of nonhomogenous Markov chains by reducing the convergence of consensus algorithm to the ergodicity of infinite products of stochastic matrices. This method is widely used in [21]–[23], [27] and others. It was based on the work of Hajnal back to the 1950’s [31]. Hajnal investigated the weak ergodicity of non-homogenous Markov chains and proposed scrambling matrix, which plays an important role in the convergence of products of stochastic matrices. Similar method has also been used to study consensus problem in continuous time networks in [17], [18], [28], etc. This method was also used to discuss synchronization in [13]. It is natural to ask if this method can be further extended to more general cases in synchronization analysis. This is the aim of this technical note. In the following, we first address global synchronization in networks with a general time-varying topology and sufficient condition for global synchronization is given. To this purpose, we extend the concept of Hajnal’s scrambling property from stochastic matrices to matrices with nonnegative off-diagonal entries. Then we will turn to stochastic switching networks. Particularly, we will study synchronization for networks with two kinds of stochastically switching topologies. That is: (a) the switching sequence are independent and identically distributed; (b) the switching sequence forms a Markov chain. In both cases, we give sufficient conditions for the network to synchronize almost surely. In previous works, the authors considered either special node dynamics such as Kuramoto model in [20], or linear dynamics such as in [13], or local synchronization as in [15]. Instead, in this note, we consider global synchronization for continuous-time networks with general nonlinear node dynamics and general time varying topologies. In case the stochastic switching topologies, we don’t require the network to switch fast enough, while this requirement is assumed in [9], [12]. We also point out that if there is a nonzero probability of a scrambling coupling matrix, then the network will synchronize almost surely if the coupling strength is strong enough. The rest of the technical note is organized as follows. In Section II, we study networks with general time-varying topologies and provide a sufficient condition for such networks to achieve synchronization. In Section III, we study stochastic network and give sufficient conditions for almost sure synchronization. An example with numerical simulations are provided in Section IV, and the technical note is concluded in Section V. II. GENERAL THEORY In this section, we discuss synchronization in networks with general time dependent coupling.

0018-9286/$26.00 © 2011 IEEE

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 57, NO. 3, MARCH 2012

Consider the following coupled dynamic system:

dxi (t) dt

=

m

f (xi (t); t) + j

=1

aij (t)0xj (t);

755

FVij (t) =

n

2 + 7!

aij (t) + aji (t) + k

6=i;j

min

+ k (t =

1

k t!+1

0

j

=1

Then by the condition lim

!+1

t

FV

j

i

0 [ FVi(s)j (s) (s)

have lim V (t) = 0, and this can complete the proof. Now we are to t!+1 prove (2). At any given time t0

which means

xki((tt )) (t0 ) =

max fxk (t ) g =1 ;...;m j (t ) xk(t ) (t0 ) = min fxki(t ) g i=1;...;m i

i

and for any j = 6 i(t0 ), j = 6 j (t0 ), we have

xki((tt )) (t0 ) xjk(t ) (t0 ) xkj((tt )) (t0 ):

=1 j 6=( ); j (t ) j i t

fai(t )j ; aj (t )j g

min

0 xjk(t ) (t0 ) 0 xkj((tt )) (t0 )

j

=1

fai(t )j ; aj(t )j g

min

6=i(t );j (t )

ai(t )j (t ) + aj (t )i(t )

xki((tt )) (t0 ) 0 xkj((tt )) (t0 )

L 0 FVi(t )j (t ) (t0 ) xik((tt )) (t0 ) 0 xkj((tt )) (t0 )

which means + V (t0 ) L 0 FV D i(t ) j (t ) (t0 ) V (t0 ):

(4)

t, synchronization analysis can be reduced to the analysis of the eigen-

0 L] ds = +1, we

V (t0 ) = xki((tt )) (t0 ) 0 xkj((tt )) (t0 )

m

)

Remark 1: In case that the coupling matrix is independent of time

(s)ds :

t

=1

+

1t!0

0 V (t) V (0)e

j

m

synchronization: < = minf i g.

+ V (t) = lim sup ((V (t + 1t) 0 V (t))=1t). If this is done, where D

Lt

=1

ai(t )j xjk(t ) (t0 ) 0 xik((tt )) (t0 )

L xki((tt )) (t0 ) 0 xkj((tt )) (t0 ) 0 k (t )

!+1 0 [ FVi(s)j (s) (s) 0

(2)

j

aj (t )j xjk(t ) (t0 )

m

)

xjk(t ) (t0 ) 0 xki((tt )) (t0 )

2

t

+ V (t) [L 0 FV (t)]V (t) D i(t)j (t)

=1

m

aj (t )j xjk(t ) (t0 ) 0 xkj((tt )) (t0 )

+ k (t

t

k

j

ai(t )j xjk(t ) (t0 ) 0

L xki((tt )) (t0 ) 0 xkj((tt )) (t0 ) 0 k(t ) ai(t )j(t ) + aj(t )i(t ) 2 xik((tt )) (t0 ) 0 xkj((tt )) (t0 )

Proof: Let y (t) = [y1 (t); . . . ; yn (t)] 2 n be such that yk (t) = i i maxi fxk (t)g 0 mini fxk (t)g. Let V (t) = ky (t)k. We will prove that

we have

m

0

faik (t); ajk (t)g:

= + , the system (1) can achieve x (t) xj (t) = 0 holds for all i; j , where 0 i

m

)

fk(t ) (xi(t ) (t0 ); t0 ) 0 fk(t ) (xj (t ) (t0 ); t0 )

+ k (t

i;j

ds lim

= 0,

t=t i(t ) = fk (t ) (x (t0 ); t0 ) 0 fk(t ) (xj(t ) (t0 ); t0 )

n

fi(t); j (t); k(t)g. In this case, we just arbitrarily pick one). Then we L]

=1 aj (t )j

m j

dt

Let xi (t) be the solution of (1) from the initial value xi (0), and j (t); k(t) be the index satisfying xki((tt)) (t) 0 xkj((tt)) (t) = let i(t); i j max kx (t) 0 x (t)k ( There may be more than one choice of can have: Theorem 1: Under Assumption 1, if lim

=

xki((tt )) (t) 0 xkj((tt )) (t)

d

i = 1; 2; . . . ; m (1)

where x 2 is the state vector, and f : is a continuous map which represents the dynamics of an uncoupled node, > 0 is the coupling strength, A(t) = [aij (t)] is the weighted outer coupling matrix that has nonnegative off diagonal entries and zero row sum for each t, and aij (t) is piecewise continuous and integrable on any finite interval of + so that standard solution of (1) exists and is unique. A(t) is also known as the negative of the Laplacian matrix of the underlying graph. 0 = diag[ 1 ; 2 ; . . . ; n ] is the weighted inner coupling matrix which is positive definite. Throughout this technical note, for x = [x1 ; x2 ; . . . ; xn ]> 2 n , kxk = max jxi j. We always make the following assumption on f : i Assumption 1: f (x; t) is Lipschitz continuous in x, with the Lipschitz constant being L > 0, i.e., kf (x; t) 0 f (y; t)k Lkx 0 y k. Let A(t) = [aij (t)] be matrices with nonnegative off-diagonal entries, for i; j 2 f1; 2; . . . ; mg, i 6= j , define a function n

i

=1 ai(t )j

m j

Differentiating and noting that we have

(3)

vectors structure of the coupling matrix. Based on the left eigenvector T = [1 ; . . . ; m ] corresponding to the eigenvalue 0, some effective synchronization criteria are given (for example, see [6], [7], where it x (t) ! 0). was proved that under some assumptions, xi (t) 0 n i=1 i i However, if the coupling matrix is time dependent, the eigenvector (t) = [1 (t); . . . ; m (t)]T corresponding to the eigenvalue 0 is also time dependent, except the coupling matrix is node-balanced. In this case, the left eigenvector corresponding to the eigenvalue 0 is (t) = T [1; . . . ; 1] , which is time independent. Therefore, all the derivation for constant coupling matrix can be used without any modifications. In this note, different from [9]–[12], [15], we propose another approach in discussing of synchronization for time-varying complex networks. Though the condition required in Theorem 1 seems difficult to verify, it is still effective for the networks with stochastically switching coupling structures. For details, see the following sections. Remark 2: When f 0, synchronization becomes consensus problem. In this case, a useful concept “ graph” was introduced in [18]. It was proved that if the connection graph satisfies that all aij (t) t+T are bounded and the graph associated to (1=T ) t A(s)ds > for

756

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 57, NO. 3, MARCH 2012

some > 0, T > 0 and all t 0 has a spanning tree with a fixed root vertex, then consensus can be realized. Based on Theorem 1, we have Corollary 1: Under Assumption 1, if

First, it is clear that when t 2 [tk01 ; tk ), FVi(t)j (t) (t) RV(Ak ). Let Kt = maxfk; tk tg, from Theorem 7.3 of [32], we can have

+

t T

1 T

min aij (t)+ aji (t)+ i;j

t

6=

lim

!1

t

minfaik (t); ajk (t)g dt >

k i;j

Then by the strong law of large numbers for i.i.d. sequence, we have

for some > 0, T > 0 and all t 0, particularly, if the coupling graph is of scrambling property for each t, i.e., for any indices i; j , i 6= j , either one of the following conditions satisfies: (1). aij (t)+ aji (t) > ; (2). There is an index k 2 f1; . . . ; ngnfi; j g such that aik (t) > > 0 and ajk (t) > > 0, then the system (1) can achieve synchronization when the coupling strength is large enough.

t

lim

!+1

t

0 FVi(s)j (s) (s)ds 0 L t

K

1

K !lim +1 t

t

Kt

t

k

In this section, we will investigate synchronization in networks with stochastically switching topology. l n ]i;j =1 2 Rn2n ; l = Let fSG ; Pg, where SG = fGl : jGl = [gij 1; . . . ; N g, P = fpl : pl > 0; l = 1; . . . ; N; N l=1 pl = 1g, be a finite sttes probability space. Furthermore, we assume that each Gl is a matrix with zero row sum and nonnegative off diagonal entries. For any A, define RV(A) = minfaij + aji + k6=i;j minfaik ; ajk gg. Then, RV (G) dei;j

l l fined by RV(Gl ) = minfgij + gji + i;j

random variable on SG . Consider

m

j

=1

k

l l 6= minfgik ; gjk gg is a

k i;j

j

aij 0x (t); t

2 [t 01 ; t k

k

);

i = 1; . . . ; m

(5)

where Ak = [akij ] 2 SG . Denote 1tk = tk 0 tk01 . In the following, we assume: (I). the sequence f1tk g is independent and identically distributed random vari~ and E f1t2k g < 1, i.e., ftn g forms a ables such that E f1tk g = 1 renewal process; (II). the switching time and topology switching are independent; (III). with probability 1, there are only finite times of switching occurs on each finite time interval, which excludes the possibility of infinitely fast switching and ensures the existence and uniqueness of the standard solution of (5).

In this subsection, we assume that fAk g is an i.i.d. sequence with probability distribution

PfAk = Gl(k) g = pl(k) ; l = 1; 2; . . . ; N:

t

t t

RV(Ak )ds

0L

t

k

k

k

k

holds almost surely. This completes the proof. B. Markov Switching Process In this subsection, we assume that the sequence fAk g is a homo= [pij ], where geneous Markov chain with the transition Matrix PfAk+1 = Gj jAk = Gi g = pij . And we assume that is irreducible, aperiodic, and has a unique stationary distribution . Similarly, we can have the following: Theorem 3: Under Assumption 1, if the expectation of RV(Gi ) with respect to the stationary distribution satisfies E fRV(Gi )g > L= , where = minf i g, the system (5) will achieve synchronization ali most surely. Proof: Similar to the proof of Theorem 2, we show t

lim

FVi(s)j (s) (s)

!+1 0

t

ds = +1:

0L

Using the fact that FVi(t)j (t) (t) RV(Ak ) when t 2 [tk01 ; tk ), and by the strong law of large numbers for Markov chains, we have

lim

!+1

t

A. I.I.D. Switching Process

=1

RV(Ak )ds +

K

t

III. MAIN RESULTS AND PROOF

t

K 1 !lim +1 t K =1 RV(A )1t 0 L 1 = EfRV(G)gEf1t g 0 L ~ 1 = EfRVg 0 L > 0 t

dxi (t) = f (xi (t); t) + dt

1 Kt = : ~ t 1

t 0 FVi(s)j (s) (s)ds 0 L

t

1

K !lim +1 t

t

Kt

t

K !lim +1 t

t

t

1

1 Kt

K

k

t

=1

t

K

k

=1

t t

RV(Ak )ds +

RV(Ak )ds

t t

RV(Ak )ds

0L

0L

E fRVgE1tk 0 L ~ !+1 1

= lim t

l(k)

2 f1; . . . ; N g depends on k. Then we have

Theorem 2: Under Assumption 1, if the expectation of N EfRV(G)g satisfies EfRV(G)g = > L= , l=1 RV(Gl )pl where = minf i g, the system (5) will reach synchronization almost i surely. Proof: From Theorem 1, it suffices to prove

P

!lim +1 0

t

t

FVi(s)j (s) (s)

0L

ds = +1 = 1:

= E fRVg 0 L > 0

holds almost surely. This completes the proof. Remark 3: It is necessary to compare the results on synchronization for switching systems obtained in this note and those reported in literature such as [9], [12], [15] and others. Briefly, in [9], [12], [15], some additional requirements on the switching speed of the network topology, the so-called ”fast-switching,” are imposed. And in [9], [15], the switching time intervals are assumed equally spaced. In such

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 57, NO. 3, MARCH 2012

757

fast enough, the switching system is ”near” to the average system in some sense. Or we can say it can approximately be the average system. Therefore, the synchronization can be reached by analyzing the average system. In this technical note, we do not require fast-switching. And compared to [9], which considers a special small- world networks generated from a regular 2 lattice, we consider a much general network structure. Yet, in this note, the coupling graph is required to have scrambling-like structure. As for [12], [15], the authors considered local stability of the synchronization manifold by linearization technique. Instead, we consider global synchronization here.

K

IV. NUMERICAL SIMULATIONS In this section, we will provide a numerical example to illustrate the theoretical results. In the following, we consider:

dx (t) = f (x (t); t) + dt i

5

i

j

a x (t); =1 t 2 [t 01 ;t ); i = 1; . . . ; 5: k ij

j

k

I

That is 0 = 3 , and assumed as

k

(6)

= 1. The dynamics of uncoupled systems are

f (x;t) = [sin(t)x; cos(t)x; sin(t + 1)x)]>

(7)

or Chua’s circuit

p[0x1 + x2 + g(x1 )] f (x; t) = x1 0 x2 + x3 (8) 0qx2 where g (x1 ) = m0 x1 + (1=2)(m1 0 m0 )(jx1 + 1j 0 jx1 0 1j). m0 = 00:68, m1 = 01:27, p = 9:0, q = 12:87. Let S = fG1 ; G2 ; G3 g, where 3 matrices 02:2889 0:8464 0:9573 0:2553 0:2299 0:6423 02:6972 0:6203 0:8586 0:5761 G1 = 0:2213 0:2789 02:2218 0:9111 0:8106 0:8371 0:7466 0:1726 02:1601 0:4038 0:9711 0:2369 0:0903 0:7252 02:0235 01:4357 0:5566 0:3178 0:2699 0:2915 0:3209 02:1481 0:4522 0:5246 0:8504 G2 = 0:5114 0:8300 03:2257 0:9727 0:9116 0:0606 0:8588 0:1099 01:6685 0:6393 0:7257 0:7890 0:1097 0:3119 01:9363 01:5879 0:5846 0:3934 0:1338 0:4761 0:8383 03:2444 0:8266 0:6715 0:9081 G3 = 0:5847 0:8277 02:5356 0:5710 0:5522 0:9481 0:1910 0:2076 01:3796 0:0329 0:0610 0:4425 0:3181 0:1477 00:9693 are picked randomly. Simple calculations show that RV(G1 ) = 1:7835, RV(G2 ) = 1:2884, RV(G3 ) = 0:6402. In the following simulations, we assume that each 1t be uniformly distributed on [0; 1]. The synchronization error is defined as err(t) = 5 1 2 =2 kx 0 x k . And the initial value are chosen randomly. G

k

i

i

A. I.I.D. Case

Fig. 1. Dynamics of x , x , x and synchronization error for the system (6) with f(x; t) being given by (7) for i.i.d. switching topology.

case, they proved that the switching system can reach synchronization (with probability 1) if the corresponding average system (or expectation system) can reach synchronization. When the switching is

In this subsection, let fA g be i.i.d. with PfA = G1 g = PfA = G2 g = PfA = G3 g = 1=3. Simulation 1: Assume that the dynamics of the uncoupled node f (x; t) is governed by (7) and = 1. Then L = 1, and EfRVg = 1:2374 > L. By Theorem 2, the system (7) can reach synchronization almost surely. The results of the simulation are provided in Fig. 1, which shows the dynamical behavior of x1 (t), x2 (t), x3 (t), i = 1, 2, k

k

k

k

i

i

i

758

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 57, NO. 3, MARCH 2012

Fig. 2. Dynamics of x (t), x (t), x and synchronization error for the system (6) with f(x; t) being Chua’s circuit (8) for i.i.d. switching topology.

Fig. 3. Dynamics of x , x , x and synchronization error for the system (6) with f(x; t) being given by (7) for Markovian switching topology.

3, 4, 5, and err(t), respectively. It can be seen that the system actually achieves synchronization.

Simulation 2: Now, we assume that the dynamics f (x; t) of the uncoupled node is governed by Chua’s circuit (8) and = 10:5. Then

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 57, NO. 3, MARCH 2012

759

B. Markov Chains In this subsection, we will consider the case that fAk g forms a Markov chain with the transition matrix

0:5311 0:3195 0:1494 = 0:5422 0:3503 0:1075 such that the unique stationary 0:5746 0:1244 0:3011 distribution = [0:5414; 0:2972; 0:1614]. Simulation 3: Let f (x; t) be given by (7), and = 1, then E fRVg = 1:4518 > 1 = L= . Therefore, By Theorem 3, the

system (7) can reach synchronization almost surely. The simulation results are presented in Fig. 3. Simulation 4: Let f (x; t) be given by Chua’s circuits (8), and = 9, then E fRVg = 9 2 1:4518 = 13:0662 > L= . By Theorem 3, the system (7) can reach synchronization almost surely. The simulation results are presented in Fig. 4. Remark 4: If we consider the following system:

dxi (t) = f (xi (t); t) + m ak 0xj (t); ij dt j =1 t 2 [tk01 ; tk ); i = 1; . . . ; m (9) where is the coupling strength and RV(G) = RV(G). By Theorem 2 and Theorem 3, we can conclude that if f satisfies the Lipschitz con-

dition, the switching topology is i.i.d. or Markovian, and the expectation EfRVg > 0 (or E fRVg > 0), then the system (9) can reach synchronization almost surely, if is large enough. Remark 5: It should be pointed out that the theoretical is quite conservative. In fact, lots of numerical experiments indicate that the system can reach synchronization even the coupling strength is much smaller than the theoretical value. For example, in simulation 4, even pick = 1, synchronization can be reached, too. We also can use adaptive algorithm to search small coupling strength in practice. V. CONCLUSION In this note, we discuss synchronization in networks with time varying coupling, i.e., the coupling structure as well as coupling weights are time dependent. Motivated by the scrambling matrix used in the study of consensus problems, we propose a novel approach without using eigen-structure of the coupling matrix. Based on the discussion of complete synchronization in networks with general time-varying coupling structure, we discuss networks with stochastically switching coupling structures. Particularly, we discuss two kinds of stochastic networks: a) independent and identically distributed switching processes and b) Markov jump processes. In both cases, we give a sufficient condition for almost sure synchronization of the networks. Also, numerical simulations are provided to illustrate the theoretical results.

REFERENCES

Fig. 4. Dynamics of x (t), x (t), x and synchronization error for the system (6) with f(x; t) being Chua’s circuit (8) for Markovian switching topology.

L = 12:87 and EfRV g = 10:5 2 1:2374 = 12:9927 > L. By The-

orem 2, the system (7) can reach synchronization almost surely. The simulation results are provided in Fig. 2.

[1] G. D. Van Wiggeren and P. Roy, “Communication with chaotic laser,” Science, vol. 279, no. 20, pp. 1198–1200, 1998. [2] M. de S. Vieira, “Chaos and synchronized chaos in an earthquake model,” Phys. Rev. Lett., vol. 82, no. 1, pp. 201–204, 1999. [3] F. C. Hoppensteadt and E. M. Izhikevich, “Pattern recognition via synchronization in phase-locked loop neural networks,” IEEE Trans. Neural Netw., vol. 11, no. 3, pp. 734–738, May 2000. [4] L. M. Pecora and T. L. Carroll, “Master stability function for synchronized coupled systems,” Phys. Rev. Lett., vol. 80, pp. 2109–2112, 1998. [5] C. W. Wu and L. O. Chua, “Synchronization in an array of linearly coupled dynamical systems,” IEEE Trans. Circuits Syst. I, vol. 42, no. 8, pp. 430–447, Aug. 1995. [6] W. L. Lu and T. P. Chen, “Synchronization of coupled connected neural networks with delays,” IEEE Trans. Circuits Syst. I, vol. 51, no. 12, pp. 2491–2503, Dec. 2004. [7] W. L. Lu and T. P. Chen, “New approach to synchronization analysis of linearly coupled ordinary differential systems,” Physica D, vol. 213, pp. 214–230, 2006.

760

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 57, NO. 3, MARCH 2012

[8] V. N. Belykh, I. V. Belykh, and M. Hasler, “Connection graph stability method for synchronized coupled chaotic systems,” Physica D, vol. 195, pp. 159–187, 2004. [9] I. V. Belykh, V. N. Belykh, and M. Hasler, “Blinking model and synchronization in small-world networks with a time-varying coupling,” Physca D, vol. 195, pp. 188–206, 2004. [10] W. Wu and T. P. Chen, “Global synchronization criteria of linearly coupled neural network systems with time-varying coupling,” IEEE Trans. Neural Netw., vol. 19, no. 2, pp. 319–332, Feb. 2008. [11] J. D. Skufca and E. Bollt, “Communication and synchronization in disconnected networks with dynamic topology: Moving neighbourhood netoworks,” Math. Biosci. Eng., vol. 1, no. 2, pp. 347–359, 2004. [12] D. J. Stilwell, E. M. Bollt, and D. G. Roberson, “Sufficient conditions for fast switching synchronization in time-varying network topologies,” Siam J. Appl. Dyn. Syst., vol. 5, no. 1, pp. 140–156, 2006. [13] C. W. Wu, “Synchronization and convergence of linear dynamics in random directed networks,” IEEE Trans. Autom. Control, vol. 51, no. 7, pp. 1207–1210, Jul. 2006. [14] W. L. Lu, F. M. Atay, and J. Jost, “Synchronization of discrete-time dynamical networks with time-varying couplings,” SIAM J. Math. Anal., vol. 39, no. 4, pp. 1231–1259, 2007. [15] M. Porfiri, D. J. Stilwell, and E. M. Bollt, “Synchronization in random weighted directed networks,” IEEE Trans. Circuits Syst. I, vol. 55, no. 10, pp. 3170–3177, Nov. 2008. [16] R. Olfati-Saber and R. M. Murray, “Consensus problems in networks of agents with switching topology and time-delays,” IEEE Trans. Autom. Control, vol. 49, no. 9, pp. 1520–1533, Sep. 2004. [17] Y. Hatano and M. Mesbahi, “Agreement over random networks,” IEEE Trans. Autom. Control, vol. 50, no. 11, pp. 1867–1872, Nov. 2005. [18] L. Moreau, “Stability of continuous-time distributed consensus algorithms,” in Proc. 43rd IEEE Conf. Decision Control, Atlantis, Paradise Island, Bahamas, Dec. 14–17, 2004, pp. 3998–4003. [19] L. Moreau, “Stability of multiagent systems with time-dependent communication links,” IEEE Trans. Autom. Control, vol. 50, no. 2, pp. 169–182, Feb. 2005. [20] A. Papachristodoulou and A. Jadbabaie, “Synchronization in oscillator networks: Switching topologies and non-homogenous delays,” in Proc. 44th IEEE Conf. Decision Control, Eur. Conf., Seville, Spain, Dec. 12–15, 2005, pp. 5692–5697. [21] A. Tahbaz-Salehi and A. Jadbabaie, “Necessary and sufficient conditions for consensus over random independent and identically distributed switching graphs,” in Proc. 46th IEEE Conf. Decision Control, New Orleans, LA, Dec. 12–14, 2007, pp. 4209–4214. [22] A. Tahbaz-Salehi and A. Jadbabaie, “A necessary and sufficient condition for consensus over random networks,” IEEE Trans. Autom. Control, vol. 53, no. 3, pp. 791–795, Mar. 2008. [23] A. Tahbaz-Salehi and A. Jadbabaie, “Consensus over ergodic stationary graph processes,” IEEE Trans. Autom. Control, vol. 55, no. 1, pp. 225–230, Jan. 2010. [24] A. Papachirstodoulou, A. Jadbabaie, and U. Münz, “Effects of delay in multi-agent consensus and oscillator synchronization,” IEEE Trans. Autom. Control, vol. 55, no. 6, pp. 1471–1477, Jun. 2010. [25] X. W. Liu, W. L. Lu, T. P. Chen, and W. Lu, “Consensus problem in directed networks of multi-agents via nonlinear protocols,” Phys. Lett. A, vol. 373, pp. 3122–3127, 2009. [26] X. W. Liu, W. L. Lu, and T. P. Chen, “Consensus of multi-agent systems with unbounded time-varying delays,” IEEE Trans. Autom. Control, vol. 55, no. 10, pp. 2396–2401, Oct. 2010. [27] S. Boyd, A. Ghosh, and B. Prabhakar, “Randomized gossip algorithms,” IEEE Trans. Inform. Theory, vol. 52, no. 6, pp. 2508–2530, Sep. 2006. [28] B. Liu and T. P. Chen, “Cosensus in networks of multiagents with cooperation and competition via stochastically switching topologies,” IEEE Trans. Neural Netw., vol. 19, no. 11, pp. 1967–1973, Nov. 2008. [29] B. Liu, W. L. Lu, and T. P. Chen, “Reaching Lp consensus in a network of multiagents with stochastically switching topologies,” in Proc. Joint 48th IEEE Conf. Decision Control 28th Chinese Control Conf., Shanghai, China, Dec. 16–18, 2009, pp. 2670–2675. [30] B. Liu, W. L. Lu, and T. P. Chen, “Consensus in networks of multiagents with switching topologies modeled as adapted stochastic processes,” SIAM J. Control Optim., vol. 49, no. 1, pp. 227–253, 2011. [31] J. Hajnal, “Weak ergodicity in non-homogenous Markov chains,” in Proc. Cambridge Philosophical Soc., 1958, vol. 54, pp. 233–246. [32] R. Durrett, Probability: Theory and Examples. Belmont, CA: Duxbury Press, 2005.

A Riccati Based Interior Point Algorithm for the Computation in Constrained Stochastic MPC Minyong Shin and James A. Primbs

Abstract—We propose a fast algorithm for the linear-quadratic control problem with probabilistic constraints that is repeatedly solved in stochastic model predictive control. Under the assumption of affine state feedback and Gaussian noise, the finite horizon control problem is converted to an equivalent deterministic problem using the mean and covariance matrix as the state. A line search interior point method is proposed to solve this optimization problem, where the step direction can be quickly computed via a Riccati difference equation. Numerical examples show that this algorithm has linear complexity in the horizon length. Index Terms—Interior point methods (IPMs), model predictive control (MPC).

I. INTRODUCTION In this technical note, we propose a fast but suboptimal algorithm for the finite horizon control of a discrete-time linear dynamic system with additive Gaussian noise under a quadratic objective and linear probabilistic constraints. While control problems of this type are of interest in their own right, their use in stochastic model predictive control (MPC) schemes accentuates the importance of fast algorithms that can be used to produce real-time solutions. Stochastic MPC is an important research area in the control community (see for example [3], [4], [10], [16], [19], [20]), and is the primary motivation for this work. Significant progress has been made in the development of Stochastic MPC methods. In particular, issues of stability and recursive feasibility have garnered attention [13], [14], and important advances have been made in the numerical on-line computations. Specifically, the numerical solution of finite horizon stochastic LQ problems with probabilistic chance constraints has received much attention, especially in the direction of developing a convex formulation. In a series of papers, van Hessem and Bosgra showed that under affine feedback assumptions it could be solved as a convex optimization problem by using a Youla parameterization [23] or “innovation feedback” [24]. Löfberg [12] and Goulart et al. [6] used the related idea of disturbance feedback to provide a convex formulation under bounded disturbances. While these convex formulations allow SDP solvers to be used, a drawback is that computation time grows quickly with problem size and horizon length. To counter this, Bertsimas and Brown [1] and Bertsimas and Sim [2] have provided tractable convex formulations in an open loop setting, and Oldewurtel et al. [18] followed up on this work with a linear programming approximation under affine feedback. The purpose of this technical note is to present a fast algorithm for the finite horizon linear-quadratic-Gaussian problem under affine state Manuscript received September 11, 2010; revised February 15, 2011; accepted June 13, 2011. Date of publication September 15, 2011; date of current version February 29, 2012. This technical note was presented in part at the American Control Conference, 2010. Recommended by Associate Editor C. Szepesvari. M. Shin is with the Department of Aeronautics and Astronautics, Stanford University, Stanford, CA 94305 USA (e-mail: [email protected]). J. A. Primbs is with the Department of Management Science and Engineering, Stanford University, Stanford, CA 94305 USA (e-mail: [email protected]). Color versions of one or more of the figures in this technical note are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TAC.2011.2168069

0018-9286/$26.00 © 2011 IEEE