The DynCOAA Algorithm for Dynamic Constraint Optimization Problems

1 downloads 448 Views 150KB Size Report
lems, as it is based upon the ant colony optimization (ACO) ... reactive principles of local search algorithms make them ..... same constraint processing engine.
The DynCOAA Algorithm for Dynamic Constraint Optimization Problems Koenraad Mertens, Tom Holvoet, Yolande Berbers AgentWise, Distrinet Department of Computer Science, K.U.Leuven 200A, Celestijnenlaan 3001 Leuven, Belgium

{Koenraad.Mertens, Tom.Holvoet, Yolande.Berbers}@cs.kuleuven.be ABSTRACT

1. INTRODUCTION

Numerous problems in software coordination, operations research, manufacturing control and others can be transformed in constraint optimization problems (COPs). Moreover, most practical problems change constantly, requiring algorithms that can handle dynamic problems. When these problems are situated in a distributed setting, distributed algorithms are preferred or even necessary. In this paper we present the dynamic constraint optimization ant algorithm (DynCOAA) that can solve dynamic COPs. DynCOAA is specifically designed for dynamic problems, as it is based upon the ant colony optimization (ACO) meta-heuristic that has already proven its merit in other dynamic optimization problems. DynCOAA is a distributed algorithm that is suited for a one-on-one mapping between variables and hosts, but it can effectively accommodate multiple variables per host. We compared our algorithm to two existing algorithms for dynamic constraint optimization: DynAWC and DynDBA. We identify a category of problems where DynCOAA outperforms the other two algorithms, while it remains competitive for other categories of problems.

Numerous problems in software coordination, operations research, manufacturing control and others can be transformed in constraint satisfaction and constraint optimization problems (CSPs and COPs). In many practical applications there is not one static CSP or COP that has to be solved: practical problems change often. Changes can happen either because additional information is added to the model, or because of a changing nature in the problem that one tries to solve. An example of a changing problem is a GPS navigation system that has to recalculate the route when the car does not follow the proposed route. When solving a changed version of a CSP or COP, the changed problem is often similar to the old problem: it only differs in a few constraints or variables. While it is possible that a small change in the problem creates a big change in the optimal solution, in most practical cases the optimal solutions of such similar problems do not differ a lot. E.g. in the GPS system when a car goes forward instead of turning left, the initial positions are almost the same and it may suffice to take the next left turn. Changing problems are called dynamic constraint satisfaction/optimization problems (DynCSP/DynCOP). They were first introduced by Dechter and Dechter in [3]. A number of centralized and distributed algorithms have been proposed for solving dynamic constraint problems. In this paper, we focus on distributed algorithms. In large, there are two categories of algorithms: complete and non-complete algorithms. Complete methods for dynamic problems are based on algorithms for static CSPs/COPs that perform some variation of a tree search. An often used adaptation to make them suited for dynamic problems consists of annotating the nogoods that are found with constraint justifications. This way nogoods that are still valid can be reused in future problems, while nogoods that have become invalid are no longer checked. Centralized and distributed variations of this technique were presented by Schiex and Verfaillie in [17] and by Modi et. al. in [15]. Non-complete methods for dynamic problems are also based upon an existing static algorithm, although the more reactive principles of local search algorithms make them more suited and easier to adapt for dynamic problems. Examples of these dynamic algorithms are the Peer-to-Peer (P2P) algorithm by Fitzpatrick and Meertens [8], based upon the static distributed stochastic algorithm (DSA) algorithm [7] and the dynamic distributed breakout algorithm

Categories and Subject Descriptors I.2.11 [Artificial Intelligence]: Distributed Artificial Intelligence—Multiagent systems

General Terms Algorithms, Design, Performance

Keywords Dynamic, ACO, Distributed Constraint Satisfaction/Optimization

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. AAMAS 2006 Hakodate, Japan Copyright 2006 ACM X-XXXXX-XX-X/XX/XX ...$5.00.

(DynDBA) by Mailler [11], that is based upon the static distributed breakout algorithm (DBA) [21]. In this paper, we present a new distributed algorithm for constraint optimization that is specifically designed for dynamic problems. It is based upon the ant colony optimization (ACO) meta-heuristic [5]. The ACO meta-heuristic has proven to be particularly useful in optimization problems such as the traveling salesman problem (TSP) and the quadratic assignment problem (QAP) [4, 9]. More recently, the ACO meta-heuristic was also applied to solve constraint satisfaction and constraint optimization problems [18, 14]. In ACO, the environment of the multi-agent system represents the problem that has to be solved and the agents use this environment to give feedback to one another. This makes ACO-based algorithms naturally suited for dynamic problem solving, as changes in the problem are reflected automatically in the environment of the agents. An example where an ACO based algorithm is used for solving the dynamic TSP problem can be found in [10]. In spite of this inherent suitability for dynamic problem solving, to this date and to the best of our knowledge, the dynamic constraint optimization ant algorithm (DynCOAA) that we present in this paper is the first distributed algorithm for dynamic constraint satisfaction or dynamic constraint optimization that uses the ACO meta-heuristic. We compare DynCOAA to two other algorithms: the complete algorithm DynAWC [20, 17, 15] and the non-complete algorithm DynDBA [21, 11]. We use two different cases for evaluating the algorithms. When the three algorithms are used to solve artificial graph coloring problems, DynDBA turns out to be the most effective algorithm. But when a real-world problem – scheduling ships that transport liquified natural gas – is tested, our DynCOAA clearly beats the other two algorithms. Next to the fact that our DynCOAA algorithm clearly has advantages over other algorithms, one of the conclusions of this paper is that a classification of problems and algorithms to solve them is necessary. In the following section we give a formal definition of a dynamic constraint optimization problem. In Sect. 3 we present our dynamic constraint optimization ant algorithm (DynCOAA). Section 4 describes the tests we performed and the results we found. Section 5 concludes this paper with an overview of our findings and some pointers for future work.

2.

DYNAMIC CONSTRAINT OPTIMIZATION PROBLEMS

In order to give a definition of a dynamic constraint optimization problem, we first define a static constraint optimization problem. A static constraint optimization problem (COP) is a tuple O = hV, D, f (V )i where ([16]): • V is a set of variables {v1 , . . . , vn }; • D is a set of domains {D1 , . . . , Dn } where each domain Di is the finite set of possible values for variable vi ; • f (A) is a cost function over the set of variables, mapping each complete variable-value assignment A to a value. The goal of solving a COP is to find the variable-value assignment A that minimizes f (A). A straightforward approach for defining the cost function is to define a number of

constraints and to assign a weight to each constraint. That way, the cost of a variable-value assignment is the sum of the weights of all violated constraints. A static constraint satisfaction problem (CSP) is a special case of a COP: the cost function is zero if no constraints are violated and infinity otherwise. A dynamic constraint optimization problem (DynCOP) is a sequence hOi i of static constraint optimization problems. The sequence can be defined by stating all problems or by giving an initial problem and a sequence of changes. In some problem areas, it is important that consecutive solutions do not differ much: when the dynamic problem of assigning landing/takeoff times and runways to airplanes – where airplanes enter and leave the radar range constantly – a time and place that was assigned to a plane should only change in exceptional circumstances. This can be included in a DynCOP by modifying the cost function fi (Ai ) to take into account the solution that was found for the previous problem: fi (Ai , Ai−1 ). Again, a dynamic constraint satisfaction problem (DynCSP) is a special case of a DynCOP.

3. DYNAMIC CONSTRAINT OPTIMIZATION ANT ALGORITHM In this section we present the dynamic constraint optimization ant algorithm (DynCOAA). We first situate the concept of this algorithm in relation to existing algorithms for both static and dynamic problems in Sect. 3.1. In Sect. 3.2 we describe how the algorithm works. Section 3.3 states the main characteristics of DynCOAA and compares them to DynAWC and DynDBA.

3.1 Situation When constraint satisfaction problems (CSPs) and constraint optimization problems (COPs) have to be solved in a distributed setting, multiagent systems (MAS) are a natural choice for architecting the system. An architecture that is often used is to map each agent to one, or a number of variables. The agents remain on their host and exchange messages to reach a solution. By specifying what kind of messages can be exchanged, privacy concerns can be met and the behavior of different kinds of centralized algorithms can be brought to the distributed platform. Examples of this kind of algorithms include the synchronous and asynchronous weak-commitment algorithm (SWC, AWC) [19, 20], the ADOPT algorithm [16], and the OptAPO algorithm [12], which are complete algorithms that use some form of tree-search to reach a solution. Noncomplete algorithms can also be distributed using a multiagent system. Examples include the distributed breakout algorithm (DB) [21] and the distributed stochastic algorithm (DSA) [7]. A different architecture is to map the application environment to the variables and to use mobile agents that move around in the environment. Algorithms that use this mapping are based upon the ant colony optimization (ACO) meta-heuristic [5]. Agents that use this meta-heuristic take the food-searching behavior of ants as a source of inspiration: they give feedback to other agents in an indirect way, by placing pheromones in the environment. Other agents detect these pheromones and modify their behavior accordingly. This meta-heuristic has already proven useful for solv-

AB

A

B

A

AB

B

A

B BA

AC

AC

CB

CA

BC

C

C

C

(a)

(b)

(c)

Figure 1: Construction of a graph for a problem with 3 variables: A ∈ {4, 5, 6}; B ∈ {2, 3}; C ∈ {2, 3} and 2 constraints: A = B + C; B > C. (a) Each main node represents a variable. (b) Going to the next main node is a two-step process: first the next variable is chosen, then the value for the current variable (A has 3 possible values: there are 3 value edges from AB to B). (c) The complete graph for the problem. For reasons of efficiency, a reference to A = B + C is stored in A, B and C, and a reference to B > C in B and C, although one reference to each constraint suffices when privacy concerns come into play in a distributed setting. ing optimization problems as the traveling salesman problem (TSP) [4]. Moreover, a changing problem is represented by a changing environment in ACO algorithms. The agents are designed to react to such changing environments, thus making these algorithms suited for dynamic problem solving [10]. In spite of this inherent suitability, to this date and to the best of our knowledge, the dynamic constraint optimization ant algorithm (DynCOAA) that we present in this paper is the first distributed algorithm for dynamic constraint satisfaction or dynamic constraint optimization that uses the ACO meta-heuristic.

3.2 Dynamic Constraint Optimization Ant Algorithm The basic idea of the ACO-meta-heuristic is that agents give positive feedback to other agents when they have found a solution. The feedback is given indirectly by placing artificial pheromones in the environment of the agents. The amplitude of the feedback depends on the quality of the solution. In order to use the ACO meta-heuristic, the problem that has to be solved must be translated into an environmental structure. In the case of solving a DynCOP with the DynCOAA algorithm, this structure is a graph, where the nodes represent variables and the edges represent values for those variables. The nodes store references to the constraints their associated variable is involved in. Additional nodes and edges are inserted to allow a more differentiated feedback. The construction of such a graph for a simple problem with three variables is shown in Fig. 1. In its simplest form, an agent that tries to solve a COP ”walks around” in the graph structure. At each node, an agent has to decide which edge to follow next. This decision is based upon the amount of pheromones that are present on each edge (feedback from previous agents). Moreover, when the cost function is composed of the sum of weights of violated constraints, each agent can check the constraints before choosing a value and thus avoids bad solutions at an early stage (as is done in a tree search). The relative importance of the feedback on the one hand and the cost function on the other hand can be tuned for each class of

problems. It is also possible to adapt the relative importances of the cost function and the feedback while solving: as more agents provide feedback, the usefull information that is provided by that feedback increases. In any case, decisions are made stochastically to prevent that after some time all agents compose the same variable-value assignment. While going from node to node, an agent remembers the path it has followed (and thus the values it has assigned to variables). When an agent has visited all nodes, it has constructed a complete variable assignment and the complete cost function can be evaluated. According to this cost, the agent puts more or less pheromones on the path it has walked. Future agents that walk the graph are influenced by pheromones that are present in the graph, but because pheromones evaporate, their influence is limited in time. At the start of the algorithm, all edges are initialized with a default pheromone quantity. As long as no agent puts pheromones on a specific edge – because no agent finds a good solution using the value associated with that edge – the pheromone quantity of that edge gradually decreases. This means that the probability that the edge gets chosen by an agent also decreases. For reasons of efficiency, not a single agent, but a swarm of agents walk the graph simultaneously. Only the agent that has found the best solution is allowed to put pheromones on the graph. The agent that has found the best overall solution is also remembered. The pheromones on the path of this best agent are increased together with that of the best agent of each swarm. When this algorithm is performed in a distributed setting, each host receives part of the graph. This allows for a one-on-one mapping between variables and hosts, but it also leaves the possibility of grouping multiple variables on each host. Different swarms of agents are executed together, each on a different host. When privacy issues are important, the agents interact with the environment to receive the value of the cost function instead of calculating that value themselves. That way, they do not have to be informed of the actual constraints and only a minimum of information is exchanged between hosts. More details about the behavior

and possibilities of this distributed algorithm can be found in [14]. When adding or removing variables or values in a dynamic problem, the corresponding nodes and edges have to be added or removed on the appropriate hosts. When adding or removing constraints, references to those constraints must be registered or deregistered with the appropriate nodes. While it is possible to leave all existing pheromone trails unchanged and only to initialize new edges with the default pheromone quantity, Guntsch et. al. show in [10] that better results can be achieved with dynamic ACO algorithms when all pheromone quantities are equalized to a certain degree. We use the Restart-Strategy that they describe in their paper, using a λR value of 0.5, which gave the best results in their experiments. This means that all pheromone quantities are set to the average of the previous pheromone quantity and the default pheromone quantity.

3.3 Algorithm Characteristics In the next section, we compare DynCOAA to DynAWC and DynDBA. While all three algorithms are suited to solve dynamic constraint optimization problems, there are some fundamental differences as to their characteristics. In contrast to DynAWC and DynDBA (and for that matter, all existing DynCOP algorithms), DynCOAA is not an adaptation of an existing static algorithm, but it has been designed from the beginning to deal with dynamic problems. The fact that the environment – a first-class entity of the solving method – represents the problem, and thus a changing problem is represented by a changing environment, in combination with the mobile agents enables this. DynCOAA and DynDBA are non-complete algorithms. This means they are not able to guarantee that the best solution eventually will be found. DynAWC, being a complete algorithm, can give that guarantee, even though it will take a very long time in any practical situation. DynCOAA and DynAWC are able to handle generalized cost functions, but they are not equally suited for it. The extreme case is when a cost evaluation is only possible when a complete variable-value assignment is available. For such cost functions, DynCOAA clearly has the best characteristics: because feedback is only given when a complete tour of the graph has been made, the feedback mechanism is not overthrown by constraints involving several or even all variables. DynAWC on the other hand loses one of its main characteristics: the ability to backtrack early when no variable can be given a value without making the cost function higher than the present optimum. DynDBA is not suited for general cost functions as it needs explicit constraint representations. It adapts the weights associated with the constraints to escape from local minima that are common when doing local search. For this reason, all tests we conducted while comparing the three algorithms used explicit constraint representations. The three algorithms differ in the amount of information, obtained during a search, they can reuse in a next search. All three algorithms are able to take the previous best solution as a guideline. DynAWC can, after a preprocessing phase, reuse the nogoods that are still valid. DynCOAA can reuse the feedback that was given in the form of pheromone quantities. DynDBA does not collect such information and thus is not able to reuse it. The adapted weights for the constraints can be reused, but we found that DynDBA has

better results when the weights are reset to their initial values. The initial value of each variable is the one that was found in the previous best solution.

4. EXPERIMENTAL RESULTS In this section we discuss the results of the tests we have done to compare DynCOAA with DynAWC and DynDBA. The three algorithms were implemented in C++, using the same constraint processing engine. We compare the algorithms for two different categories of problems. The first category of problems, discussed in Sect. 4.1, is artificial graph coloring problems. In Sect. 4.2 we discuss the results of a real-world problem, in which a schedule for a number of ships that transport liquified gas must be assembled. Section 4.3 gives an interpretation of the obtained results.

4.1 Graph Coloring Tests For our first series of tests, we use the same problems as can be found in [11], i.e. graph coloring problems with 30 variables, which require a coloring with 3 colors. Graph densities varied over {2.0; 2.3; 2.5}. These densities represent the three different phases of the phase transition for random 3-coloring graphs (respectively mostly satisfiable; within the transition; and mostly unsatisfiable, see [2]). The number of changes between successive problems, indicated by δ, varied over {2; 4; 6; 8; 12}, where 6 changes means that 3 constraints were removed from the problem and 3 constraints added to the problem, so that the overall density remained unchanged. For each combination of graph density and change rate δ, 20 problems were constructed, each consisting of an initial COP and 100 successive COPs. In contrast to the tests performed in [11], the DynCOPs we tested explicitly required successive solutions to be similar, because this is a precondition in many real-world cases. Therefore, we changed the cost function of the individual COPs to include extra constraints, requiring each variable to have the same value as in the previous best solution. Because the weight of these extra constraints was only 0.3 (with a weight of 1 for the normal constraints), approximately 3 variables could change their value for each constraint violation that could be resolved. As proposed in the future work section of [11], we measured the real-time performance of the algorithms: for each successive COP, the algorithms could calculate during 2 seconds. The best solution at each moment in time was stored. This form of measurement provides a more realistic indication of the performance than can be obtained through cycle-based tests. DynCOAA can be configured using a number of parameters: the number of agents N in each swarm, the amount of pheromones η dropped by an agent, the sensitivity of an agent to a pheromone trail (expressed by the osmotropotaxic sensitivity β and the sensor capacity 1/δ), the evaporation rate κ of the pheromones and the reset-value γ and resetdegree λR of the Restart-Strategy. Instead of tuning all these parameters, which would take much time, or just randomly picking any value, we used values that, according to the literature that describes them, should give good results. When the algorithm is effectively used, the parameters can be tuned with e.g. a genetic algorithm. For these tests, we used the values N = 5, η = 0.07, β = 3.5, δ = 0.2, κ = 0.015, γ = 0, and λR = 0.5. These values and their exact meaning can be found in [1] (parameters η, β, δ, κ,

Density

2.0

2.3

2.5

δ 2 4 6 8 12 2 4 6 8 12 2 4 6 8 12

DynCOAA First Solution Best Solution Time Cost Cost 0.13 3.99 2.42 0.13 4.85 3.00 0.14 5.47 3.37 0.13 6.27 4.02 0.13 7.35 4.79 0.13 5.43 3.67 0.14 6.21 4.25 0.14 6.73 5.49 0.13 7.51 5.05 0.13 8.90 6.18 0.13 6.42 4.53 0.15 7.43 5.26 0.17 8.14 5.87 0.14 8.71 6.21 0.14 10.01 7.25

DynAWC First Solution Best Solution Time Cost Cost 0.053 19.24 8.48 0.044 19.86 9.29 0.044 20.70 9.84 0.039 21.05 9.71 0.054 22.08 10.61 0.055 22.65 12.23 0.056 23.36 12.20 0.060 24.09 12.89 0.049 24.62 13.29 0.047 25.42 13.64 0.057 25.76 14.70 0.058 26.13 15.06 0.057 26.85 15.63 0.043 27.50 15.76 0.057 28.79 16.59

DynDBA First Solution Best Solution Time Cost Cost 0.034 1.90 1.40 0.034 2.65 1.87 0.032 3.35 2.23 0.034 3.69 2.25 0.034 4.51 2.48 0.025 3.07 2.60 0.028 3.61 2.84 0.026 4.33 3.23 0.025 4.77 3.37 0.027 5.60 3.59 0.018 4.07 3.59 0.028 4.88 4.09 0.027 5.45 4.33 0.036 5.87 4.46 0.028 6.70 4.70

Table 1: Results of testing DynCOAA, DynAWC and DynDBA on 30-node, dynamic 3-coloring problems. Averages over 20 problem instances per Density-δ combination. Time is expressed in seconds. and γ) and [10] (parameter λR ). The number of agents N in the swarm was chosen to be 5, according to the relatively small graph size of 30. Table 1 gives an overview of the results of the three algorithms. The table includes the average time (in seconds) that passed before all 20 problems found the first solution, the average cost of that first solution and the average cost of the best solution that was found after 2 seconds. The average costs of the first solution of DynDBA are better than the errors for the same algorithm, on the same type of problems, that are reported in [11]. This despite the fact that in our tests, the problems had additional similarity constraints. The reason for this seemingly contradiction is the longer running time of the algorithm. In [11], the algorithm could only compute for 1 cycle one each successive problem, while we allowed it to compute for 2 seconds (which equals about 50 to 100 cycles). As can be seen in the table and the anytime curve, the improvement that is realized between the first solution and the best solution after 2 seconds is relatively large. With this in mind, it is easy to see that an algorithm that can start from a better previous solution will also result in a better first solution. Another, graphical view of some of the results can be found in Fig. 2, 3 and 4. These figures are anytime curves. They relate the computed time of an algorithm to the average cost of the best solutions. A thin line means that the algorithm has not found a solution yet for all of the 20 problems at the given time. In the figures, the anytime curves are given for the initial problem and changes number 10, 50 and 100. This is done for the problems with a change rate δ = 6 and a density of 2.0 (Fig. 2), 2.3 (Fig. 3), and 2.5 (Fig. 4) When looking at the table and the anytime curves, some interesting observations can be made. First of all, DynCOAA needs about 5 times longer to calculate its first solution than is necessary for DynDBA and DynAWC. This is due to the fact that DynCOAA uses a swarm of 5 agents, who all need about the same amount of time. Secondly, when we look at the anytime curves, we see that when solving the initial problem, DynDBA reaches an av-

erage solution of nearly zero. Despite this very good result, it takes DynDBA a relatively long time (0.1 to 0.3 seconds, or about 5 to 15 cycles) to improve itself after a change has taken place. This is represented by the horizontal first part of each of the anytime curves at changes 10, 50 and 100. This is probably due to a local minimum caused by a too specified solution for the previous solution. Despite this initial delay, DynDBA still produces the best solutions after 2 seconds of calculation. Thirdly, for the initial problem, DynAWC is better than DynCOAA. The average best solutions it finds are only a little worse than the ones found by DynDBA, while the average best solutions of DynCOAA have a substantial higher cost. But when we look at the behavior of DynCOAA and DynAWC during changes, DynCOAA clearly beats DynAWC, even though it takes about 5 times longer to find its first solution. Fourthly, the difference between the average best solutions of DynDBA and DynCOAA is smaller than the difference between DynCOAA and DynAWC. As a conclusion, we can say that for these dynamic graph coloring problems, DynDBA is the best algorithm, closely followed by DynCOAA.

4.2 Real-World Ship Scheduling Tests For our second series of tests, we decided to use a realworld problem instead of an artificial problem. This realworld problem is to assemble a schedule for transport ships. The ships transport liquified natural gas from different ports around the world to one destination port. Using data provided by Tractebel Engineering [6], the algorithms had to calculate a schedule for 7 ships, for a period of three months. After a decent schedule had been found, we simulated a storm: one of the ships got delayed for three days. The algorithms had to recalculate the schedule, incorporating the delay. The goal was twofold: to obtain a good schedule, but also to leave the individual schedules for the 6 ships that did not get stuck in the storm as unchanged as possible. This problem is quite complicated: the original scheduling problem is made up of a total of 69 variables and 101

25

DynCOAA DynAWC

0

5

Average Cost of Solution 10 15 20

DynDBA

Original 0.01

Change #10 0.1

1

10 0.01

0.1

Change #50 1

10 0.01 Seconds

0.1

Change #100 1

10 0.01

0.1

1

10

30

Figure 2: Anytime curves for DynCOAA (solid line), DynAWC (dashed line) and DynDBA (dotted line) when solving graph coloring problems with 30 variables, a density of 2.0 and a δ of 6. DynCOAA DynAWC

0

Average Cost of Solution 10 20

DynDBA

Original 0.01

Change #10 0.1

1

10 0.01

0.1

Change #50 1

10 0.01 Seconds

0.1

Change #100 1

10 0.01

0.1

1

10

Figure 3: Anytime curves for DynCOAA (solid line), DynAWC (dashed line) and DynDBA (dotted line) when solving graph coloring problems with 30 variables, a density of 2.3 and a δ of 6. constraints. The variable domains contain up to 200 possible values. 76 constraints are hard constraints (those have to be obeyed in each valid solution: it are constraints with a weight of infinity) and 25 are soft constraints, used for optimizing the cost function. 57 of the hard constraints are binary constraints. All other hard constraints involve at least 19 variables. The soft constraints even involve all variables. A solution with a cost of 1000 means that the local transport rate in the destination port has to deviate from the optimal rate during a total of one day (in smaller intervals distributed over the three months). At a cost of 2000, it has to deviate during a total of 2 days, etc. Each of the three algorithms made 20 schedules. For the original schedule as well as for the storm-adaptation, 10 minutes of calculation were allowed. The parameters that were used for the DynCOAA are the same as described in Sect. 4.1, except for the swarm size, which was increased to 30. The anytime curves for this problem are shown in Fig. 5. The first thing to notice is that DynDBA is missing from Fig. 5. During the 10 minutes of calculation time, none of the 20 runs were able to find any valid schedule (one that satisfies all hard constraints). The most probable reason

for this is that a number of the constraints are global constraints, making it very hard for a local search algorithm to find a solution. Because no initial schedule was found, DynDBA could not even begin to search for a solution after the change. The two algorithms that did find a valid solution did so in all 20 runs. But the behaviors of DynCOAA and DynAWC were completely different for the original and the changed problem. For the original schedule, it took both algorithms about 1 minute to find a first solution for all 20 runs. The average solution of DynCOAA was twice as good as the average solution of DynAWC at that time. The average solution of DynAWC quickly became better and after 10 minutes DynAWC had found an almost perfect solution in all 20 runs, while DynCOAA still had a cost of about 550. As for the schedule after the storm, it took DynCOAA less than 10 seconds to find a valid schedule in all 20 runs. Even more, the first average solution was almost as good or even better than the last average solution before the storm. DynCOAA could further improve this first schedule to end at an average cost of about 300 after 10 minutes of calculation. DynAWC on the other hand took almost 100 seconds

30

DynCOAA DynAWC

0

Average Cost of Solution 10 20

DynDBA

Original 0.01

Change #10 0.1

1

10 0.01

0.1

Change #50 1

10 0.01 Seconds

0.1

Change #100 1

10 0.01

0.1

1

10

8000

Figure 4: Anytime curves for DynCOAA (solid line), DynAWC (dashed line) and DynDBA (dotted line) when solving graph coloring problems with 30 variables, a density of 2.5 and a δ of 6. DynCOAA

0

Average Cost of Solution 2000 4000 6000

DynAWC

Original 1

Change 1 10

100

1000 1 Seconds

10

100

1000

Figure 5: Anytime curves for DynCOAA (solid line) and DynAWC (dashed line) when assembling a schedule for ships. DynDBA was unable to find any solution. to find a first solution in all 20 runs. The first average solution had a cost of about 5000. In the remaining time, the solution could be improved to less than 2000, but this still was a lot higher than the average solution of DynCOAA. One of the reasons of the slower response rate of DynAWC is the preprocessing of the nogoods. After 10 minutes of calculation for the initial schedule, more than 10.000 nogoods had been accumulated. It took DynAWC about 10 seconds to filter out the nogoods that had become invalid after the change. By that time, DynCOAA had already found a new solution.

4.3 Evaluation The results of DynDBA are the most dissimilar when we look at the behavior over the two different categories of tests. For the artificial graph coloring problems, it was the clear winner of the three algorithms. For the real-world ship scheduling problems, it could not find any solution. DynCOAA and DynAWC on the other hand behave rather similar for the two different categories of problems. For the first, static problem, DynAWC is a better choice.

But when the static problem becomes a dynamic problem by changing some constraints, DynCOAA reacts faster to those changes and finds a better solution. As a result, we can identify a category of problems where DynCOAA outperforms the other tested algorithms: dynamic constraint problems that include both a combination of satisfaction and optimization (hard and soft constraints), and of binary and global constraints. Because it is possible to do cost evaluations on a global level with DynCOAA, it is better for this category of problems than DynDBA, which is designed for local constraints. The reactive nature of the ACO meta-heuristic makes that DynCOAA outperforms DynAWC in dynamic problems. A more thourough categorization of problems and algorithms is not available at this time.

5. CONCLUSIONS AND FUTURE WORK In this paper we presented DynCOAA. It is based on the ACO meta-heuristic and designed from the start for solving dynamic constraint optimization problems (DynCOPs). We compared this algorithm against two other algorithms in

two different categories of problems: artificial graph coloring problems and real-world ship scheduling problems. There are two important conclusions that can be drawn from this comparison: • DynCOAA is suited for solving DynCOPs, as it beats DynAWC in all two and DynDBA in one of the two categories of problems.

[11]

• Choosing the right algorithm for a specific problem is very important, because the performance of algorithms can depend greatly on the type of problem. This second conclusion is one that definitely needs further attention. It is our intention to make a classification of problems, categorized by how well they can be solved by DynCOAA. Similar classifications should be made for other algorithms as well.

6.

ACKNOWLEDGMENTS

[12]

[13]

This work has been funded by the Institute for the Promotion of Innovation through Science and Technology in Flanders (IWT-Vlaanderen) [14]

7.

REFERENCES

[1] D. R. Chialvo and M. M. Millonas. How Swarms Build Cognitive Maps. In L. Steels, editor, The Biology and Technology of Intelligent Autonomous Agents, volume 144, pages 439–450. Nato ASI Series, 1995. [2] J. Culberson and I. Gent. Frozen development in graph coloring. Theoretical Computer Science, 265(1–2):227–264, 2001. [3] R. Dechter and A. Dechter. Belief maintenance in dynamic constraint networks. In Proceedings of the Sixth National Conference on Artificial Intelligence (AAAI ’88), pages 37–42, 1988. [4] M. Dorigo and L. M. Gambardella. Ant Colony System: A Cooperative Learning Approach to the Traveling Salesman Problem. IEEE Transactions on Evolutionary Computation, 1(1):53–66, April 1997. [5] M. Dorigo, V. Maniezzo, and A. Colorni. Positive Feedback as a Search Strategy, Technical Report 91016, Dipartimento di Elettronica e Informatica, Politecnico di Milano, Italy, 1991. [6] T. Engineering. http://www.engineering.tractebel.com/. [7] S. Fitzpatrick and L. Meertens. An experimental assessment of a stochastic, anytime, decentralized, soft colourer for sparse graphs. In proceedings of the First Symposium on Stochastic Algorithms: Foundations and Applications, pages 49–64, 2001. [8] S. Fitzpatrick and L. Meertens. Distributed coordination through anarchic optimization. In V. Lesser, C. L. Ortiz, Jr., and M. Tambe, editors, Distributed Sensor Networks: A Multiagent Perspective, pages 257–295. Kluwer Academic Publishers, 2003. [9] L. Gambardella, E. Taillard, and M. Dorigo. Ant Colonies for the QAP. Journal of the Operational Research Society, 50:167–176, 1999. [10] M. Guntsch, M. Middendorf, and H. Schmeck. An ant colony optimization approach to dynamic TSP. In L. Spector, E. D. Goodman, A. Wu, W. B. Langdon,

[15]

[16]

[17]

[18]

[19]

[20]

[21]

H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. H. Garzon, and E. Burke, editors, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), pages 860–867, San Francisco, California, USA, 7-11 2001. Morgan Kaufmann. R. Mailler. Comparing two approaches to dynamic, distributed constraint satisfaction. In Proceedings of the Fourth International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS ’05), pages 1049–1056, Utrecht, The Nederlands, July 25-29 2005. R. Mailler and V. Lesser. Solving distributed constraint optimization problems using cooperative mediation. In Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS ’04), pages 438–445, New York, New York, USA, July 19-23 2004. K. Mertens and T. Holvoet. CSAA; a Constraint Satisfaction Ant Algorithm Framework. In Proceedings of the Sixth International Conference on Adaptive Computing in Design and Manufacture (ACDM’04), pages 285–294. Springer-Verlag, 2004. K. Mertens and T. Holvoet. CSAA; a Distributed Ant Algorithm Framework for Constraint Satisfaction. In Proceedings of the 17th International FLAIRS Conference, pages 764–769. AAAI Press, 2004. P. J. Modi, H. Jung, M. Tambe, W.-M. Shen, and S. Kulkarni. A dynamic distributed constraint satisfaction approach to resource allocation. In CP ’01: Proceedings of the 7th International Conference on Principles and Practice of Constraint Programming, pages 685–700, London, UK, 2001. Springer-Verlag. P. J. Modi, W.-M. Shen, M. Tambe, and M. Yokoo. An asynchronous complete method for distributed constraint optimization. In Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS ’03), pages 161–168, Melbourne, Australia, July 14-18 2003. T. Schiex and G. Verfaillie. Nogood Recording for Static and Dynamic Constraint Satisfaction Problem. International Journal of Artificial Intelligence Tools, 3(2):187–207, 1994. C. Solnon. Ants can solve constraint satisfaction problems. IEEE Transactions on Evolutionary Computation, 6(4):347–357, August 2002. M. Yokoo. Weak-Commitment Search for Solving Constraint Satisfaction Problems. In Proceedings of the 12th National Conference on Artificial Intelligence (AAAI ’94); Vol. 1, pages 313–318, Seattle, WA, USA, July 31 - August 4 1994. AAAI Press, 1994. M. Yokoo. Asynchronous Weak-Commitment Search for Solving Distributed Constraint Satisfaction Problems. In Proceedings of the First International Conference on Principles and Practice of Constraint Programming (CP ’95), pages 88–102. Springer-Verlag, 1995. M. Yokoo and K. Hirayama. Distributed Breakout Algorithm for Solving Distributed Constraint Satisfaction Problems. In Proceedings of the Second International Conference on Multiagent Systems, 1996.