Parallel and Distributed Evolutionary Computation with MANIFOLD Franciszek Seredynski1 (
[email protected]), Pascal Bouvry2 (
[email protected]), Farhad Arbab2 (
[email protected]) 1 Institute of Computer Science, Polish Academy of Sciences
Ordona 21, 01-237 Warsaw, Poland
2 Centre for Mathematics and Computer Science (CWI)
Kruislaan 413, 1098 SJ Amsterdam, The Netherlands
Abstract. In this paper, we apply a competitive coevolutionary ap-
proach using loosely coupled genetic algorithms to a distributed optimization of the Rosenbrock's function. The computational scheme is a coevolutionary system of agents with only local interaction among them, without any central synchronization. We use a recently developed coordination language called Manifold to implement our distributed optimization algorithm. We show that the distributed optimization algorithm implemented using Manifold outperforms the sequential optimization algorithm based on a standard genetic algorithm.
1 Introduction Recent years have witnessed a growing interest in the design of multi-agent systems to solve real life problems. The algorithms developed on a multi-agent platform are characterized by a high degree of parallelism and distributed control, which decreases the complexity of the algorithms and increases their eciency. However, to eciently apply the multi-agent systems methodology, some theoretical and practical issues must be resolved. Among them, in particular, are the two questions (a) what should be the nature of an agent taking a decision and how agents should interact to solve a problem, and (b) how to eciently implement multi-agent systems. In this paper we address both these questions. In the next section we describe a game-theoretic model of interaction between agents, and propose an evolutionary technique called loosely coupled genetic algorithms to implement an agent's behavior. In Section 3 we describe a recently developed coordination language called Manifold, which we use to implement a multi-agent system. Section 4 presents a problem of optimization of the Rosenbrock's function through our evolutionary techniques, and Section 5 describes an implementation of a distributed algorithm to solve this problem using Manifold. We show in Section 6 the performance of our proposed approach and compare it with the performance of a standard genetic algorithm. The last section contains our conclusions.
2 Competitive Coevolutionary Systems 2.1 Games with Limited Interaction
A game with a limited interaction is given by a set N = f0; 1; : : :; N ? 1g of N players, a set Sk of actions for each player k 2 N , and a payo function uk for each player k 2 N which depends only on its own action sk and the actions of its Nk neighbors in the game. The game can be represented by an oriented graph G =< V; E > called an interaction graph. V is the set of nodes corresponding to the set of players. The set E represents the pattern of interaction between players: incoming arcs to the k-th node de ne players whose actions in uence the payo of player k. It is assumed that each player acts in the game independently and selects his action to maximize his payo . Such a model belongs to the class of non-cooperative games and has been primarily considered from stand point of collective behavior of learning automata. The most widely used solution concept for non-cooperative games is the Nash equilibrium point. A Nash point is an N -tuple of actions, one for each player, such that anyone who deviates from it unilaterally cannot possibly improve his expected payo . A Nash equilibrium point de nes payo s of all the players in the game. We are interested in some global measure of the payo s received by players. This measure, called price of a game s, can be, e.g., the average payo u(s) received by a player as a result of the combined actions s = (s0 ; s1 ; : : :; sN ?1 ), i.e., !
u(s) =
NX ?1 k=0
uk (s) =N:
(1)
The question which arises immediately concerns the value of the criterion (1) at a Nash point. Unfortunately, this value can be very low. Analyzing all possible action combinations in a game and evaluating their prices, i.e., the u(s) values, we can nd action combinations characterized by a maximal price and we can call them maximal price points. Maximal price points are action combinations which maximize the global criterion (1), but they can be reached by players only if they are Nash points. A maximal price point usually is not a Nash point and the question which must be resolved is how to convert a maximal price point into a Nash point. To solve this problem we introduce the concept of cooperation between players, by allowing them to share their payo s in a game. We use two basic schemes of cooperation:
{ global cooperation: sharing a payo received by a player k with all players
participating in the game, i.e., his payo uk is transformed into a new payo wk as (2) uk ?! wk = u(s) { local cooperation: sharing a payo received by a player k with his neighbors in the game, i.e., his payo is transformed as
P
uk + l2Nk ul uk ?! wk = max ; (3) l2N Nk + 1 where ul is the payo of a neighbor l of the player k, Nk is the number of neighbors of the player k, and maxl2N Nk denotes the maximal number of neighbors in a game. We are interested in a global behavior (in the sense of the criterion(1)) of a team of players taking part in iterated games. To evolve such a behavior we will use an evolutionary computation technique based on genetic algorithms (GA) [7].
2.2 Competitive Coevolutionary Multi-Agent Systems To support the game-theoretic model of computation described above, loosely coupled GAs (LCGAs) representing a competitive coevolutionary approach have been proposed [13]. LCGAs can be speci ed in the following way: #1: for each player create a subpopulation of its actions: create for each player an initial random subpopulation of size sub pop size of player actions taking values from the set Sk of its actions. #2: play a single game: in a discrete instant of time, each player randomly selects one action from the set of actions prede ned in its subpopulation that is not used until now, and presents it to its neighbors in the game. calculate the output of each game by evaluating each player's local payo uk in the game. #3: repeat step #2 until sub pop size games are played. #4: for each player create a new subpopulation of its actions: after playing sub pop size games each player knows the value of its payo received for a given action from its subpopulation. the payo s are considered as values of a local tness function de ned during a given generation of GA; standard GA operators of selection, crossover, and mutation are applied locally to subpopulations of actions; these actions will be used by players in games played in the next game horizon. #5: return to step #2 until the termination condition is satis ed.
3 Manifold: A Coordination Language for Massively Concurrent Activities An LGCA optimization algorithm describes how to orchestrate a set of GA-based agents, each using some local cost function, in order to achieve the maximal value for a global cost function. The Manifold coordination language is particularly well suited to describe such an orchestration of processes running concurrently on a multi-platform computing environment.
Manifold is based on a generic model of communication that supports the separation of computation and coordination responsibilities, and encourages a weak dependence of workers (processes) on their environment. We refer to this generic model as the Idealized Worker Idealized Manager (IWIM) model. More detailed description of this model is given elsewhere [2]. The basic concepts in the IWIM model are processes, events, ports, and channels. A process is a black box with well de ned ports of connection through which it exchanges units of information with the other processes in its environment. A port is a named opening in the bounding walls of a process through which units of information are exchanged using standard I/O type primitives analogous to read and write. We use the notation p:i to refer to the port i of the process p. The interconnections between the ports of processes are made through channels. We write p:o ?! q:i to denote a channel connecting the port o of the producer process p to the port i of the consumer process q. A process in IWIM can be regarded as a worker process or a manager (or coordinator) process. The responsibility of a worker process is to perform a (computational) task. A worker process is not responsible for the communication that is necessary for it to obtain the proper input it requires to perform its task, nor is it responsible for the communication that is necessary to deliver the results it produces to their proper recipients. In general, no process in IWIM is responsible for its own communication with other processes. It is always the responsibility of a manager process to arrange for and to coordinate the necessary communications among a set of worker processes.
4 Competetive Coevolutionary Approach to Function Optimization In the evolutionary computation literature [8, 9, 16] there are a number of test functions which are used as benchmarks for contemporary optimization algorithms. The Rosenbrock's function:
f2 (x) =
n X i=1
? 100 x2i ? xi+1 2 + (1 ? xi )2
(4)
for x 2 IRn and ?2:12 xi 2:12 is an example of a function that is a real challenge (see, e.g. [5, 8, 10, 11, 12]) for any optimization algorithms. This function has its global minimum f2 (x ) = 0 at x = (1; 1; : : :; 1). While attempting to nd a global optimum, a diculty arises from the fact that the optimum is located in a deep parabolic valley p with a at bottom. Fig. 1 (upper) shows a transformation g(x1; x2) = 5 ? f2 (x1 ; x2) of the Rosenbrock's function giving an insight into the topology of the function for the case when n = 1. Fig. 1 (bottom) shows the contour of the transformation with a visible parabolic valley and the optimum area. Fig. 2 shows a computational scheme used to nd a minimum of the Rosenbrock's function with standard GAs. A solution x = (x1 ; x2; : : :; xn+1) called an
g(x1,x2) -40-30-20-10 0 10 1 x2 1.5 2 2
2.0
00 .5
1 x1
1.5
0.5
0
1.5
-7-6-5-4-3-2-101234
x2 1.0
4.8 4.8 4.8 4.8
0.0
0.5
4.8
4
0.0
3 2 1 0 -1-2-3-4-5-6-7
0.5
1.0
1.5
2.0
x1
Fig. 1. Response surface of the Rosenbrock's function transformation (upper), and the contour of the function transformation (lower)
n
f2(x) =Σ (100(x 2i -x i+1) +(1-x )i ) 2
2
i=1
x1
x2
xn+1
Fig. 2. Optimization of Rosenbrock's function with GA individual is coded as a binary string, and a population of individuals is created. A tness function for each individual fjl in a given generation l is evaluated according to the value of f2 (x). To apply the GA methodology, the problem of minimization of the Rosenbrock's function is converted into the problem of maxl ? fl, imization by the modi cation of the tness of an individual: fjl = Cmax j l where Cmax is the tness of the worst individual in a generation l. Next, GA operators are applied to create a population of new solutions. This evolutionary process is continued for some number of generations, and is terminated after converging to a solution. In a standard GA, a solution x = (x1; x2; : : :; xn+1) called an individual is coded as a binary string, and a population of individuals is created. A tness function for each individual is evaluated according to a value of f2 (x). Next, GA operators are applied to create a population of new solutions. The evolutionary process is continued for a number of generations, and is terminated after converging to a solution. We consider the problem of minimization of f2 (x) as a problem of seeking a minimum in a distributed fashion. We use a multi-agent system with a gametheoretic model of interaction among agents, with the interaction graph shown in Fig.3. The multi-agent system performing the distributed optimization is based 1
f2(x 1,x2) A1
n-1
2
f2(x2,x3) A2
...
f2 (xn-1 ,xn )
...
A n-1
n
f 2(xn,xn+1) An
Fig. 3. Interaction graph for the Rosenbrock's function evaluation agents on LCGAs and is implemented using Manifold.
Each agent Ai has information only about its locally de ned function ? f2i (xi; xi+1) = 100 x2i ? xi+1 2 + (1 ? xi)2 ;
(5)
which depends only on its xi and the xi+1 of its neighbor i+1. This local function is minimized by an agent during a game, while the performance of the whole game is evaluated through the value of the global criterion function (4). Individuals in a subpopulation corresponding to a given agent i (i = 1; 2; : : :; n? 1) code solutions for a variable xi. The agent n runs two subpopulations corresponding to variables xn and xn+1. Individuals in subpopulations are evaluated according to local functions. Only a monitor of the distributed optimization algorithm knows the global state of the optimization process.
5 Implementation In our implementation of the LGCA-based optimization algorithm using Manifold, the agent behaviour is described in a classical sequential language, e.g., C, while the interconnection scheme is described in the Manifold language. The role of this coordinator is rst to instantiate the atomic workers and install the communication streams among them, and nally to activate them. Coordinator
111 000 000 111 000 111 out in 000 111 000 111
Monitor
111 000 000 111 000 111 out in 000 111 000 111
tags:
111 000 000 111 out in 000 111 000 111
out
1
out
out
11 00 00 11 in 00 11 00 11
out i-1
2
tag
tag
events
111 000 000 111 out in 000 111 000 111 tag
...
Agents
out
111 000 000 111 out in 000 111 000 111
out
i
tag
n
11 00 00 11 in 00 11 00 11 tag
...
out
Fig. 4. Manifold implementation of the Rosenbrock's interaction graph. In Fig. 4, a global snapshot of the running processes of the Manifold program is given. The set of processes consists of the coordinator, the monitor and the di erent agents coupled with their tags. The Rosenbrock's interaction graph (see, Fig. 3) is implemented as a linked list of di erent agents (see, Fig. 4). Here is described the construction of the agent list in the Manifold language:
Initialization:
Process first is Agent.nn The first agent is instantiated. Process previous is Agent.nn The loop previous element is instantiated. previous=&first.nn The pointer previous is initialized by the address of
the first agent.
Loop:
for i=2 while i
