Planning problem Additional difficulties - CiteSeerX

Planning in Multiagent Systems

Goal of this tutorial

Mathijs de Weerdt and Cees Witteveen

• Overview of • multiagent planning problems • multiagent planning techniques

EASSS, May 8, 2008 1 EASSS, May 8, 2008

Faculty of Electrical Engineering, Mathematics and Computer Science, Department of Software Technology

2

CWI, Amsterdam, Sen-4, Han la Poutré

Tutorial Contents

Outline

Part I: 1. 2. 3.

• Introduction to multiagent planning • Relation with multiagent systems • Relation with planning

(Mathijs) Introduction Taxonomy of problems Taxonomy of techniques

• Taxonomy of multiagent planning problems

Part II: Plan Coordination Mechanisms and their Evaluation (Cees) 1. Overview of plan coordination in MAP 2. Design and evaluation of coordination methods 3. Applications of coordination methods EASSS, May 8, 2008

• Classical Planning • Taxonomy of multiagent planning techniques • Recent focus • Discussion 3

EASSS, May 8, 2008

Planning problem

Additional difficulties

• How to get from the current state to your goal state?

Because the world is … • Dynamic • Stochastic • Partially observable

• Planning involves • Action selection • Action sequencing • Resource handling

And because actions • take time • have continuous effects • involve multiple agents!

• Plans can be • Action sequences • Policies/strategies (action trees)

EASSS, May 8, 2008

4

5

EASSS, May 8, 2008

6

Why planning in multiagent systems?

Multiagent Planning (MAP) and Multiagent Systems (MAS)

• More efficient system performance on run-time • Come prepared • Prevent deadlock • Lower costs • Accomplish task more quickly

• MAS: Coordination of autonomous entities • MAP ⊆ MAS (hence this tutorial) • MAP: Focus on • Coordination of actions before execution • Finding correct actions to attain goals

• Useful assignment of resources, use of capabilities

• Strongly related to task allocation and auction/negotiation techniques EASSS, May 8, 2008

8

UN Operations

MAP and AI planning

Applications of Multiagent Planning

• MAP ⊇ AI Planning for multiple agents • Execution in parallel (instead of sequentially) → Parallel plans

• Planetary explorations

• MAP ⊇ Planning by multiple agents (distributed) • Incoherent plans: need for coordination; more difficult, less optimal • Why then? • Privacy & autonomy • Local → more efficient reaction on incidents (when communication limited) & no central point of failure • Speed-up: in parallel & smaller problems

• Taxi companies

EASSS, May 8, 2008

• Multi-player video games

9

Planetary explorations

EASSS, May 8, 2008

Multi-player video games

• More or less independent

• Self-interested players

• Communication difficult and costly

• Independent

• Cooperative

• Potential for coordinated behavior

EASSS, May 8, 2008

10

11

EASSS, May 8, 2008

12

Taxi companies

Outline • Introduction to multiagent planning • Relation with multiagent systems • Relation with planning • Taxonomy of multiagent planning problems • Classical Planning • Taxonomy of multiagent planning techniques • Recent focus • Discussion

EASSS, May 8, 2008

13

EASSS, May 8, 2008

14

Strongly related → Loosely coupled → Independent

Taxonomy of MAP problems

Strongly related because

Four ways to look at multiagent planning problems

• Joint actions • Limited shared resources

• Strongly related → Independent

PhD research

Requires crisp coordination

• Cooperative → Self-interested • (Resolving conflicts → Exploit efficiency)

Examples

• No communication → Reliable communication

EASSS, May 8, 2008

15

• • • • •

Lift a box together Car assembly Robocup Hospital PhD research Car assembly

Cooperative → Self-interested Independent

PhD research Rovers

UN-operation Hospital

Robocup Strongly related

EASSS, May 8, 2008

Resolving conflicts → Exploit efficiency

Lifting boxes Cooperative

Independent

Warcraft

Warcraft Taxi companies

Hospital

Human soccer Supply chains

Self-Interested / Private 17

PhD research

Traffic

Traffic

Strongly related

EASSS, May 8, 2008

Lifting boxes Resolve conflicts

Rovers Taxi companies

Robocup Supply chains

Both

Exploit efficiency 18

Continuous planning and execution

No communication → reliable communication

• Any system that is used should deal with unexpected events • Plan adaption (repair) or • Start from scratch (replanning)

Examples: • Rescue Robots • Planetary explorations • Military operations

EASSS, May 8, 2008

• Multi-agent planning can help here by locally replanning / plan repair • when no communication possible • to reduce the impact of incidents 19

EASSS, May 8, 2008

Outline

Aim of AI


• “Intelligent” systems, decide themselves • what to do, and • how to do it.


• Planning is… • given what to do (goals), • determine how (and when) to do it (plan).

• Classical Planning • Taxonomy of multiagent planning techniques

20

• Currently planning research community very active • Significant scale-up • Bi-annual planning competition

• Recent focus • Discussion EASSS, May 8, 2008

21

AI Planning background

EASSS, May 8, 2008

22

Classical Planning

• Focus on classical planning; assume none of the above • Deterministic, static, fully observable • “Basic” • Most of the recent progress • Ideas often also useful for more complex problems

• Classical planning • Model • STRIPS • Refinement planning framework • Partial plans • Plan space refinement • HTN planning • Complexity of planning

EASSS, May 8, 2008

23

EASSS, May 8, 2008

24

Classical planning model

Planning is searching

• Origins: • STanford Research Institute Problem Solver (’71) • derived from GPS = human problem solving (’61)

…in state space (or…)

I want to be here! or hereor here

Choose between possible actions • Depth-first • Breadth-first

• States described by propositions currently true

goal states

• Actions: general state transformations described by sets of pre- and post-conditions ?

• Represents a state-transition system (but more compact) EASSS, May 8, 2008

I’m here!

a plan is a route in state space

25

initial state

Searching for a plan

STRIPS Formalism (now in PDDL)

start from initial state, try all possible actions

• action: preconditions, add, delete effects

→ large search space

• pickup(B1, B2) • precondition: empty & clear(B1) &

STRIPS: regression: look at goal state!

on(B1, B2)

• add-effect: holding(B1), clear(B2) • delete-effect: empty, on(B1, B2),

clear(B1)

• problem: initial state, actions/operators, goal description • objects and variables EASSS, May 8, 2008

28

I’m here!

STRIPS algorithm

STRIPS demo

STRIPS( s, g )

(by CI – space, British Columbia (CA))

returns: a sequence of actions that transforms s into g

Action

Preconditions

Add List

Delete List

1. Calculate the difference set d=g-s. 1. If d is empty, return an empty plan

pickup(B1, B2)

empty & clear(B1) & on(B1, B2)

holding(B1), clear(B2)

empty, on(B1, B2), clear(B1)

2. Choose action a whose add-list has most formulas contained in g

pickuptable(B)

empty & clear(B) & ontable(B)

holding(B)

empty, ontable(B), clear(B)

3. p’ = STRIPS( s, precondition of a )

putdown(B1, B2)

holding(B1) & clear(B2)

empty, on(B1, B2), clear(B1)

clear(B2), holding(B1)

4. Compute the new state s’ by applying p’ and a to s.

putdowntable(B)

holding(B)

empty, ontable(B), clear(B)

holding(B)

5. p = STRIPS( s’, g ) 6. return p’;a;p

g p s’ a p’ s

STRIPS planner

Classical planning in a multiagent setting

• Questions? • Optimal? • Sound? • Complete?

• When an agent is unable to do a task • Give task to other • Other: adapt current plan → plan repair Van der Krogt (2005)

EASSS, May 8, 2008

31

EASSS, May 8, 2008

Refinement planning framework

Refinement planning template

• Framework to capture all planning algorithms

Refineplan( P : Plan set)

• Idea: Narrow set P of potential action sequences

1.

If P is empty, Fail.

• Subcontents: • Specify action sequences by partial plans • Refinement strategies • Generic template

2.

If a minimal candidate of P is a solution, return it. End

3.

Select a refinement strategy R

4.

Apply R to P to get a new plan set P’

5.

Call Refineplan(P’ )

32

Termination ensured if R complete and monotonic. EASSS, May 8, 2008

33

EASSS, May 8, 2008

34

Existing Refinement Strategies

Plan space refinement (I)

• State space refinement: e.g. STRIPS

• Least commitment planning (Weld, 94) • search in plan space instead of state space • represent plans more flexible: not a sequence, but a partially ordered set • keep track of decisions and the reasons for these decisions

• Plan space refinement: e.g. Least commitment planning • Task refinement: e.g. HTN

EASSS, May 8, 2008

35

EASSS, May 8, 2008

36

Partial Plans: Syntax Partial plan = (Actions, partial Ordering, causal Links) - causal links = Interval preservation constraint (IPC) (a1 , p , a2) - p must be preserved between a1 and a2

Plan space A state in the (plan) search space is a partial plan (instead of a description of the state of the world)

a4 (a3, q, a4) a2 a3 a1

EASSS, May 8, 2008

37

My plan: I’m here!

POP { goal description }

• Deal with one (g,a) at a time

a1: move b2 from table to b3

b2

¬clear(b2)

a2: move b1 from table to b2

b3

clear(b2) { initial state }

EASSS, May 8, 2008

b3 b1

a0

b2

{ on(b3,b1), clear(b2), on(b1,table), on(b2, table) }

a0 39

POP( (A,O,L), agenda )

EASSS, May 8, 2008

40

Tradeoffs among Refinements

1.

Termination: if agenda is empty return (A,O,L)

2.

Goal selection: select a (g,aneed) from the agenda

3.

Action selection: choose an action aadd that adds g. Update L, O, and A

4.

Update goal set: remove (g,aneed) from agenda, and add its preconditions (…, aadd)

5.

Causal link protection: for every action at that might threaten a link, add an ordering constraint

EASSS, May 8, 2008

{on(b1,b2), on(b2,b3)}

b1

a∞ • Start with • null (empty) plan • agenda = list of (precondition, action) goals = {(g1, a∞), (g2, a∞), (g3, a∞), …}

a∞

POP example

41

State space refinement: • commit to both order and relevance of actions • include state information (easier plan validation) • lead to premature commitment • too many states when actions have durations

EASSS, May 8, 2008

Plan-space refinement: • commit to actions, avoid constraining order • increase plan-validation costs • reduce commitment (large candidate set /branch) • easily extendible to actions with duration

42

HTN Planning

Partial plans in a multiagent setting

task

travel(x,y)

travel by taxi

OR

• Broadcast (abstraction of) part of your plans relevant for others → “partial global plan”

AND

• Keep updating this global plan until nothing changes

airport(x,a) airport(y,b) ticket (a,b) travel (x,a) fly(a,b) travel(b,y)

method

travel by air

travel(Delft, Cap.) airport(Delft,AMS) airport(Cap.,LIS) ticket(AMS,LIS) travel(Delft, AMS) get taxi ride taxi(Delft, AMS) pay driver fly(AMS, LIS) travel(LIS, Cap.) get taxi ride taxi(LIS, Cap.) pay driver

Problem reduction

Generalized Partial Global Planning (Durfee, Decker, Lesser, et al.)

EASSS, May 8, 2008

43

•

Decompose tasks into subtasks

•

Handle constraints (e.g., taxi not good for long distances)

•

Resolve interactions (e.g., take taxi early enough to catch plane)

•

If necessary, backtrack and try other decompositions

AND

get taxi ride taxi (x,y) pay driver

Hierarchical multiagent planning

Complexity of planning

• Task structure for all involved agents • QAFs • SUM (~ AND), SyncSUM • MAX (~ OR), MIN • Non-local effects • Enables, Disables • Facilitates, Hinders

• PSPACE-completeness (NP⊆PSPACE) • PLANEX: plan existence problem is PSPACE-complete for propositional STRIPS • Restrictions on preconditions allowed and on effects, helps to reduce complexity

• TAEMS (Decker), c_TAEMS for Coordinators (Boddy et al.) • Zlot and Stentz (2006) EASSS, May 8, 2008


45

•

plan length is at most l=2n

•

hardness: each PSPACE problem can be translated by making Turing-machine operations into actions

•

plan existence with non-deterministic Turing machine in O( log l ) space (polynomial in n) plan(s,g) 1. if s=g return true 2. guess intermediate state s’ 3. return plan(s,s’) and plan(s’,g)

EASSS, May 8, 2008

46


complete

PSPACE PLANEX is PSPACE-complete

EASSS, May 8, 2008

pre: preconditions

complete

* pre * effs

effs: effects

NP

* pre 1 effs

P

1 pre * effs

PSPACE-complete

2 +pre 2 effs

NP-complete

* pre * +effs 1 pre 1 effs

O(log l)

Polynomial 47

1 +pre 1 effs

1 pre * effs k goals

0 pre * effs

Future work

Recommended reading

Apply other planning techniques to multiagent planning: • Non-deterministic actions (surprises) • Partially observable world (exact costs unknown) • Durative actions (move, travel) and continuous variables (capacity, fuel, distance, time) → optimality

• Kambhampati, S. (1997). Refinement planning as a unifying framework for plan synthesis. AI Magazine, 18(2):67-97.

EASSS, May 8, 2008

49

Other references •

• Ghallab, M., Nau, D., Traverso, P. (2004). Automated Planning: Theory and Practice, Elsevier. • Vlahavas I., Vrakas, D. (2005). Intelligent Techniques for Planning, Idea Group.

EASSS, May 8, 2008

50

Outline

Boddy, M., Horling, B., Phelps, J., Goldman, R. Vincent, C. Long and B. Kohout, C_TAEMS Language Specification, Version 1.06. Decker, K. S. and Lesser, V. R. (1992). Generalizing the partial global planning algorithm. International Journal of Intelligent and Cooperative Information Systems, 1(2):319-346. Decker, K. (1996). TAEMS: A Framework for Environment Centered Analysis & Design of Coordination Mechanisms. Foundations of Distributed Artificial Intelligence, Chapter 16, G. O'Hare and N. Jennings (eds.), Wiley Inter-Science, pp. 429-448.


•

Fikes, R. E. and Nilsson, N. (1971). STRIPS: A new approach to the application of theorem proving to problem solving. Artificial Intelligence, 5(2):189-208.


• •

Newell, A. and Simon, H. (1963). GPS: A program that simulates human thought. In Feigenbaum, E. and Feldman, J., editors, Computers and Thought, pages 279-296. Van der Krogt, R. and de Weerdt, M. (2005). Coordination through Plan Repair. In Gelbukh and de Albornoz and Terashima-Marin (Eds.). MICAI 2005: Advances in Artificial Intelligence, pages 264-274. LNAI 3789.

•

Weld, D. S. (1994). An introduction to least-commitment planning. AI Magazine, 15(4):27-61.

•

Zlot, R. M. and Stentz, A. (2006). Market-based multirobot coordination for complex tasks. International Journal of Robotics Research, Special Issue on the 4th International Conference on Field and Service Robotics, 25(1):73-101.

• •

EASSS, May 8, 2008

51

• Classical Planning • Taxonomy of multiagent planning techniques • Recent focus • Discussion EASSS, May 8, 2008

52

Taxonomy of MAP techniques

Where are plans created?

• Where are plans created? • Centralized • Distributed

Algorithm: Centralized • Optimal (potentially) 1. Label actions with agent • Communication only names twice (before & after) 2. Plan

• When coordination in MAP process? • Before planning • During planning • Post-planning

3. Decompose into subplans

• How are plans coordinated? • Dependency checks? • Task/resource allocation? EASSS, May 8, 2008

4. Add synchronization actions 53

EASSS, May 8, 2008

54



Examples: Distributed • Ephrati (1995): by • Reduce computation plan merging time • Keep privacy (potentially) • vd Krogt (2005): by plan repair • Scalable • Execution and control are also distributed

Partially distributed • Share parts of your plan

EASSS, May 8, 2008

• Durfee, Decker, Lesser (1986-) Partial global planning (shared plan) 55

Where are plans created? Distributed, for a centralized plan • Specialized planning agents

EASSS, May 8, 2008

56

When coordination in MAP process?

Examples: • Kambhampati (1991): Combining specialized reasoners and planning

1. Global task refinement = AI planning 2. Task allocation 3. Coordination before planning 4. Individual planning

• Wilkins (1998): Multiagent planning architecture EASSS, May 8, 2008

Examples: • Corkill (1979): Distributed NOAH (shared world model)

5. Coordination after planning 6. Plan execution 57

EASSS, May 8, 2008

When coordination in MAP process? (pre)

When coordination in MAP process? (during)

3. Coordination before planning (pre-planning coordination) • Social laws (e.g. traffic rules): Shoham & Tennenholtz (1992) • Derive specific constraints for agents: Buzing, Valk et al. (2006)

4. Individual planning Coordination during planning • Distributed NOAH (Corkill, 1979) • Partial global planning (Decker, 1992) • Through plan repair (vd Krogt, 2005)

EASSS, May 8, 2008

EASSS, May 8, 2008

59

58

60

When coordination in MAP process? (post)

When coordination in MAP process? (continual)

5. Coordination after planning • Plan merging (Ephrati, 1993)

• Plans are being executed during planning and coordination • May break and re-make commitments • unexpected event/failure • goal change

6. Coordination during (plan) execution • When communication is unreliable (Koes, 2006) • “Normal” MAS/distributed systems solutions • FIFO-queues • Semaphores

EASSS, May 8, 2008

(DesJardins, 2000)

Distributed continual planning Task allocation During-planning Execution Task refinement Pre-planning Post-planning coordination coordination 61

EASSS, May 8, 2008

Difficulties of continual planning

How are plans coordinated?

• Chain reactions of changes

• Maintenance of dependencies? • Distributed check for cycles • Shared global plan • Afterwards: plan merging

• Cyclic dependencies ?

62

• Distribution of tasks and resources? • Contract net protocol • Auctions (combinatorial?)

EASSS, May 8, 2008

63

EASSS, May 8, 2008

Taxonomy of MAP

Outline

Problems: • Strongly related → Independent • Cooperative → Self-interested • (Resolving conflicts → Exploit efficiency) • No communication → Reliable communication

• Introduction to multiagent planning • Relation with multiagent systems • Relation with planning • Taxonomy of multiagent planning problems

Techniques: • Where are plans created? • Centralized → Distributed • When coordination in MAP process? • Before planning → During planning → Post-planning • How are plans coordinated? • Dependency checks? • Task/resource allocation? EASSS, May 8, 2008

64

• Classical Planning • Taxonomy of multiagent planning techniques • Recent focus • Discussion 65

EASSS, May 8, 2008

66

Recent focus

Self-interested planning agents

• Mixed initiative multiagent planning • using domain knowledge (in task networks)

Problem setting • Planning routes for internet packages along selfinterested routers

• More application-oriented work: • search & rescue • military operations (“Coordinators”) • logistics

• Coordinating transport companies in a supply chain • As a government, set up a market for coordinated medical treatments, or public transport

• Self-interested agents • Probabilistic planning: Markov-Decision Processes

→ mechanism design!

• Distributed Constraint Optimization EASSS, May 8, 2008

67

EASSS, May 8, 2008

68

Mechanism design

Multiagent planning as a social choice

• Sub-field of economic theory from engineering perspective

• Given a set of n agents that each have their own goals and actions. Find a multiagent plan that is “good” for every agent (Pareto-efficient).

• Assuming agents are rational, find a way to obtain a social choice (joint decision), eg • elections • markets • auctions • government policy • Apply the theory…

Q. Is it possible to find a mechanism to solve this problem for selfinterested agents? How/why not? A. No. Gibbard-Satterthwaite (’73) say: • Agents will have incentive to lie about preferences (or operations), or • social choice will be a dictatorship. Maybe introduce money or tax to incentivize agents…

EASSS, May 8, 2008

69

EASSS, May 8, 2008

Mechanisms with money

Mechanisms with money

• Def: Vickrey-Clarke(’71)-Groves(’73) VCG mechanism: assign payments p1,…,pn to each agent such that • f maximizes social welfare: f(v1,…,vn) ∈argmaxa∈A’Σivi(a) • payment pi does not depend on vi and is related to the sum of values for all other players: pi(v1,…,vn) =-Σj ≠ivj(f(v1,…,vn)) + hi(v-i)

Goal. Find a multiagent plan that optimizes Σivi. Q. Is it possible to find a mechanism to solve this problem for selfinterested agents (if vi is private information) ? How/why not? A. Yes, using a VCG-based mechanism, but: • we retrieve some “tax”; what to do with that? • social choice must be optimal, general planning is PSPACEcomplete and if not optimal then cost minimization allocation problem with possibly very bad results • lying about actions may lead to infeasible solutions: another penalty afterwards?

• VCG theorem: every VCG mechanism is incentive compatible. EASSS, May 8, 2008

70

71

EASSS, May 8, 2008

72

Problems, problems, …

Recommended reading

• How to determine the valuations for the agents when planning is PSPACE-complete? May also depend upon other agents’ utilities (inter-dependent utilities)

• Noam Nisan (2007). Ch. Introduction to Mechanism Design (for computer scientists) in Algorithmic Game

Theory.

• Multiagent planning problem has often on-line context • Sometimes we cannot use a central, but need a distributed mechanism

EASSS, May 8, 2008

73

Other references •

K. Arrow. Social Choice and Individual Values. Yale University Press, 1951. E. H. Clarke. Multipart pricing of public goods. Public Choice, pages 17–33, 1971.

•

Allan Gibbard. Manipulation of voting schemes: a general result. Econometrica, 41:587–601, 1973.

•

T. Groves. Incentives in teams. Econometrica, pages 617–631, 1973. Philippe Jehiel and Benny Moldovanu. "Efficient Design with Interdependent Valuations," Econometrica, Econometric Society, vol. 69(5), pages 1237-59, 2001.

•

David Parkes. Ch. Online Mechanism Design in Algorithmic Game Theory, 2007.

•

K. Roberts. The characterization of implementable choice rules. In Aggregation and Revelation of Preferences, J-J. Laffont (ed.), North Holland Publishing Company., 1979.

•

Mark Allen Satterthwaite. Strategy-proofness and Arrow’s condition: Existence and correspondence theorems for voting procedures and social welfare functions. Journal of Economic Theory, pages 187–217, 1975.

•

W. Vickrey. Counterspeculation, auctions and competitive sealed tenders. Journal of Finance, pages 8–37, 1961.

EASSS, May 8, 2008

• Privacy & autonomy • First negotiating, then (optimal) planning • No central point of failure • Just introduce redundancy • Limited communication (on execution) • Contingent planning on forehand & with constraints on-line • Speed-up: in parallel & smaller problems • Use a grid and parallel computing techniques

75

Recommended reading

EASSS, May 8, 2008

76

Other references • •

• Durfee’s chapter (3) on Distributed Problem Solving and Planning in Weiss (1999), Multiagent systems: A modern approach to

• •

Distributed Artificial Intelligence

• DesJardins, M. E., Durfee, E. H., Ortiz, C. L., and Wolverton, M. J. (2000). A survey of research in distributed, continual planning. AI Magazine, 20(4):13-20. • (Links to) material can be found on: http://www.st.ewi.tudelft.nl/~mathijs/tutorial.php

• • • • •

EASSS, May 8, 2008

74

Discussion: Reasons for not doing multiagent planning

•

•

EASSS, May 8, 2008

77

Buzing, P., ter Mors, A., Valk, J. and Witteveen, C., Coordinating Self-interested Planning Agents, Autonomous Agents and Multi-Agent Systems 12(2):199-218. Corkill, D. (1979), Hierarchical Planning in a Distributed Environment. In Proceedings of the Seventh International Joint Conference on Artificial Intelligence, pages 168-175. Decker, K. S. and Lesser, V. R. (1992). Generalizing the partial global planning algorithm. International Journal of Intelligent and Cooperative Information Systems, 1(2):319-346. Ephrati, E. and Rosenschein, J. S. (1993). Multi-agent planning as the process of merging distributed sub-plans. In Proceedings of the Twelfth International Workshop on Distributed Artificial Intelligence (DAI-93), pages 115-129. Kambhampati, S., Cutkosky, M., Tenenbaum, M., and Lee, S. (1991). Combining specialized reasoners and general purpose planners: A case study. In Proceedings of AAAI-91, pages 199-205. Koes, M., K. Sycara, and I. Nourbakhsh (2006), A Constraint Optimization Framework for Fractured Robot Teams. In AAMAS. Shoham, Y. and Tennenholtz, M. (1995). On social laws for artificial agent societies: Off-line design. Artificial Intelligence, 73(1-2):231-252. van der Krogt, R. and de Weerdt, M. (2005). Coordination through Plan Repair. In Gelbukh and de Albornoz and Terashima-Marin (Eds.). MICAI 2005: Advances in Artificial Intelligence, pages 264-274. LNAI 3789. Wilkins, D. and Myers, K. (1998). A multiagent planning architecture. In Proc. of the 4th Int. Conf. on AI Planning Systems (AIPS), pages 154-162.

EASSS, May 8, 2008

78

Plan Coordination Mechanisms and their evaluation

Cees Witteveen Faculty EEMCS, Delft University of Technology [email protected] Plan Coordination and Autonomy

Planning in Multi Agent Systems Plan Coordination and Autonomy

EASSS 2008

EASSS 2008

1

2

disasters

disasters

coordination in preparation

What’s the difference coordination in action Plan Coordination and Autonomy

Plan Coordination and Autonomy

EASSS 2008

team sport

3

overview of the talk (i)

4

overview of the talk (ii) Part II

Part I

we discuss an application to multi-modal transportation planning and show that in some cases - !optimal coordination can be approximated very efficiently - !the coordination method can be used to reuse single agent ! planning technology in a multi-agent context

we present an overview of plan coordination in MAP discussing factors relevant to coordination and types of coordination in planning we concentrate on the design of coordination methods suitable for selfish agents that are not able or willing to revise their autonomously developed plans

next, we show how aspects of time can be incorporated by discussing a coordination method ensuring autonomous temporal planning autonomy. finally, we discuss an application to taxi-routing on airports discussing a sequential reservation-based planning approach offering agents to plan autonomously given the knowledge about resource usage of other agents

to evaluate the quality of coordination methods we discuss the price of autonomy measuring the quality loss of planning using autonomous planning. Plan Coordination and Autonomy

EASSS 2008

team sport


EASSS 2008

5

EASSS 2008

6

problems we want to deal with airport planning

Part I

planning systems for arrival, departure, gate assignment, ground handling taxi-route planning

overview of plan coordination in

patient planning planning systems for treatments of patients, making reservations for resources

multi-agent planning

multi-modal logistics planning planning systems for traveling & transportation using different transportation modalities Plan Coordination and Autonomy


EASSS 2008

EASSS 2008

7

common characteristics

8


catering

several autonomous planners each have to perform a disjoint set of tasks

arrive ! taxi ! dock ! cleaning ! taxi ! depart fueling

tasks might be interdependent (within and across agents) intake ! specialist1! scan ! specialist2 ! surgery

to avoid conflicts each agent has to construct a plan for its set of tasks. each plan has to satisfy the given ordering constraints and might impose new constraints.

truck - train - plane - truck - ship - truck



EASSS 2008

EASSS 2008

9

10

Plan coordination : preliminaries


First: Preliminary definition

several autonomous planners each have to perform a disjoint set of tasks

plan coordination refers to a collection of methods to obtain a set of jointly feasible plans for a set of goals, given agents who

• •

tasks might be interdependent (within and across agents)

want to achieve a specific subset of these goals by making plans individually

REQUIRES: PLAN COORDINATION to avoid conflicts each agent has to construct a plan for its set of tasks.

Next: Preliminary questions

each plan has to satisfy the given ordering constraints and might impose new constraints.


•

what are the main characteristics of the Multi-Agent Planning (MAP) process relevant to coordination?

•

what are the main types of plan coordination given this MAP process?


EASSS 2008

11

EASSS 2008

12

Multi Agent Planning: process view tasks

Multi Agent Planning: process view

agents

tasks

agents

tasks consisting of interrelated subtasks and specifications of capabilities

set of agents together with description of capabilities.

task allocation

task allocation task allocation: process of assigning tasks to agents such that capabilities offered by agents are sufficient for capabilities required by tasks.

planning

planning planning by individual agents; result should be a feasible joint plan

environment

environment plan execution


plan execution


EASSS 2008

EASSS 2008

13

factors influencing coordination properties dependencies

Properties of Tasks

capabilities relationships

tasks

14

•

loosely coupled

•

moderately coupled

•

tightly coupled

agents

environment

no (dependency) relations between subtasks assigned to different agents

dependency relations between subtasks assigned to different agents

relations between tasks that require close interaction during execution of the plan

resource availability Plan Coordination and Autonomy


EASSS 2008

EASSS 2008

15

Properties of Environment

Properties of Agents •

cooperative agents are willing to revise and to adapt their plans to the plans of other agents

•

willing/able to communicate

•

self-interested requiring planning autonomy

agents are willing and able to exchange plan information

agents want to plan autonomously, are not willing/able to share information or to revise plans upon request.


16

•

resource availability

•

execution interaction

•

stability

do the agents have to compete for resource usage or not

does the execution of tasks negatively (hinders) or positively (facilitates) influence the execution of other tasks

is the environment stable or incident prone and dynamic.


EASSS 2008

17

EASSS 2008

18

Plan Coordination methods tasks

Coordination methods

agents

tasks

agents

coordination before planning task allocation

task allocation

coordination during planning

planning

environment

planning



plan execution


EASSS 2008

EASSS 2008

19

Coordination methods tasks

20

coordination: types, goals, methods

agents

coordination type

goals

methods

after planning

conflict resolution

plan merging plan revision

during planning

conflict resolution conflict prevention

plan communication plan revision

before planning

conflict prevention

constraint management

task allocation

planning

coordination after planning




EASSS 2008

EASSS 2008

21

22

coordination: types, goals, methods coordination type

goals

methods

after planning

conflict resolution

plan merging plan revision

during planning

conflict resolution conflict prevention

plan communication plan revision

before planning

conflict prevention

constraint management


Part I

designing plan coordination mechanisms


EASSS 2008

23

EASSS 2008

24

a simple construction example

problem: general statement

There are two agents A1 and A2 involved in a construction task. A1 has to deliver bricks to A2, who has to use them to build a wall. A2 has to ensure that garbage will be collected and has to deliver it to A1 who will collect it and bring it to a dumping ground.

Construct a coordinated plan for a set T of interdependent tasks distributed over a set of autonomous agents A, where the agents enjoy planning autonomy, that is the agents ! are free in making their own plans for their set of tasks

t1

! do not want or are not able to exchange information during the planning process ! are not willing to revise their plan in composing

A1

a feasible joint plan for the set of tasks

transport garbage

pickup garbage


task level

t2

deliver bricks

build wall

t6

A2

collect garbage

t4

t5


EASSS 2008

t3

deliver garbage

EASSS 2008

25

26

task and plan level 1. drive to construct- 5. load bricks ion place; 6. drive to construct2. pickup garbage ion place 3. drive to dumping 7. deliver bricks place 4. take coffee PLAN of A 1

tasks and plans

PLAN of A2

t1

t2

deliver bricks

A1

transport garbage

pickup garbage

1. drive to construct- 5. load bricks ion place; 6. drive to construct2. pickup garbage ion place 3. drive to dumping 7. deliver bricks place 4. take coffee PLAN of A1

1. take coffee; 5. take coffee; 2. build part wall; 6. collect garbage 3. take coffee; 7. deliver garbage 4. build part wall

t3


collect garbage

t2

deliver bricks

A2

A1

transport garbage

t4

t5

PLAN of A1

t1 build wall

t6


deliver garbage

pickup garbage

build wall

t6

A2

collect garbage

t4

t5


EASSS 2008

t3

deliver garbage

EASSS 2008

27

tasks and plans

28

tasks and plans

1. drive to construct- 5. load bricks 1. take coffee; 5. take coffee; ion place; 6. drive to construct2. build part wall; 6. collect garbage 2. pickup garbage ion place 3. take coffee; 7. deliver garbage 3. drive to dumping 7. deliver bricks relationship Task-Plan 4. build part wall place 4. take coffee A1 PLAN ofofA1 induceof partitioning on set plan steps • tasksPLAN


• plan step ordering has to satisfy task ordering t1 t2 of wall deliver bricks • plan step ordering may induce refinement build

A1

task ordering.


PLAN of A1

INFEASIBLE JOINT PLAN t1

t2

deliver bricks

build wall

A2

additional constraints induced by plans

A1

transport garbage

pickup garbage

t6

t3

A2

transport garbage

t4

t5


collect garbage

deliver garbage

pickup garbage

t6

29

collect garbage

t4

t5


EASSS 2008

t3

deliver garbage

EASSS 2008

30

coordination: during / after

recapitulation: our framework partially ordered set of tasks

autonomous agents

A1

A2

1. drive to construct- 5. load bricks ion place; 6. drive to construct2. pickup garbage ion place 3. drive to dumping 7. deliver bricks place 4. take coffee PLAN of A

where tasks require different capabilities

with different capabilities

A3 COORDINATION PROBLEM

1

Ensure that a feasible joint plan for the set of A2to tasks is produced if eachA1 agent wishes plan individually.

A1

A3

PLAN of A2

t1

t2

deliver bricks

transport garbage

build wall

t6

t3

A2

collect garbage

t4

t5

pickup garbage

task assigment


deliver garbage

dependencies between agents



EASSS 2008

EASSS 2008

31

32

coordination: during / after 1. drive to construct- 5. load bricks ion place; 6. drive to construct2. pickup garbage ion place 3. drive to dumping 7. deliver bricks place 4. take coffee PLAN of A 1

A1

NEW PLAN of A2

t1

transport garbage

pickup garbage

t2 build wall

revision and communication required

t3

1. take coffee; 2. collect garbage; 3. take coffee; 4. build wall 5. deliver garbage

PLAN of A1 Coordination before planning?

A2

A1

t1

t2

deliver bricks

collect garbage

transport garbage

deliver garbage

pickup garbage

t4

t5



1. take coffee; 2. collect garbage; 3. take coffee; 4. build wall 5. deliver garbage

deliver bricks

t6

coordination

build wall

t6

t3

collect garbage

t4

t5

deliver garbage


EASSS 2008

A2

EASSS 2008

33

coordination problem: definitions

34

example

example, just for illustration

Planning instance Is this instance coordinated ?

A plan coordination instance is a tuple ( [ T1, T2, . . ., Tk ] , < ) where each Ti is a set of tasks given to agent Ai. The tuple ( Ti, lab(x).

•

as a consequence, every inter-agent path is monotonically label-increasing => no inter-agent cycle is possible => plan coordination achieved.

traversing an intra-agent edge u < v, the label setting algorithm ensures lab(u) ! lab(v).

1

for all tasks t, t’ ! TA do if lab(t) < lab(t’) then add ( t < t’ ) to
example Plan Coordination and Autonomy


EASSS 2008

47

EASSS 2008

48

Multi-modal transportation Part II

A1

A2

A3

A6

coordination design and price of autonomy in multi-modal planning

A5

A4

order is a sequence of simple transportation tasks.

cost of plan is measured by #moves + #(un)load actions

we are given n of those orders Plan Coordination and Autonomy


EASSS 2008

EASSS 2008

49

designing coordination mechanism • •

50

the price of autonomy: theoretical results paut =

applying the label setting algorithm results in a nearly optimal coordination mechanism with approximation ratio " 2

sum cost optimal local transportation plans

=

cost optimal total transportation plan

1.14

We know that finding an optimal local transportation plan is difficult. There is a simple 2-approximation algorithm for these problems:

significance

peff aut =

approximation algorithm decomposes complex multi-agent planning problem into several independent single-agent planning problems: existing single agent planning technology can be re-used

sum cost approximated local transportation plans cost optimal total transportation plan

=

1.22

=

1.11

The best approximation algorithm for the optimal solution realizes: popt =

sum cost approximated total transportation plan cost optimal total transportation plan



EASSS 2008

EASSS 2008

51

52

plan cost comparison

paut : empirical results

500 We performed two types of experiments using an AIPS logistic benchmark set

375 250

plan cost

minimum plan STAN coordination TAL planner HSP

comparing the coordination approach (= coordination + local heuristics) with best state-of-the art planners.

125 comparing state-of-the art planners with state-of-the-art planners as single agent planners + the coordination mechanism.

0 20

30

40

50

60

nr of orders Plan Coordination and Autonomy


EASSS 2008

53

70

EASSS 2008

54

using planners as single agent planners STAN time

100%

STAN #steps

HSP time

Autonomy vss Efficiency

HSP #steps

minimizing makespan and independent planning

80%

• suppose each subtask t in a complex task T j

has a given duration dj

savings

60%

• we are interested in both independent planning by

40%

the agents as well as minimization of the makespan of the plan

20%

results 0% 1

6

11

16

21

26

31

36

41

46

• minimal set of constraints can be computed efficiently while

51

garanteeing independent planning and minimal makespan

-20%

problem



EASSS 2008

EASSS 2008

55

56

Autonomy vss Efficiency idea compute time constraints for scheduling tasks of agents

Part II

• for every task t in T compute its depth d(t) and its height h(t) • set the following constraints (d(t), h(T) -h(t) +1)

coordination design

• for every t, t’ such that t < t’, remove overlap

in temporal planning

results

• every set of locally satisfiable schedules s(t) constitutes a globally minimal schedule



EASSS 2008

EASSS 2008

57

coordination in temporal planning •

•

how to deal with time windows

adding time information •

quantitative time information: time windows for tasks

•

qualitative time information: timing relations between tasks

representing time windows For every task ti a time window [ tlb , tup ] is provided.

problem to solve we have to ensure that for every set of individual plans chosen by the agents, a feasible joint schedule can be found

plan coordination

method

how to ensure that autonomous planning agents come up with independently chosen plans such that there always exist a feasible schedule for the joint plan satisfying the temporal constraints.


58

[19,45]

[20,50]

t1

t2

t3

t4

[19,55]

[18,55]

we reduce the problem to standard plan coordination by eleminating time windows. The set of time windows is represented by an additional agent executing “critical timepoint tasks”


EASSS 2008

59

EASSS 2008

60

removing time windows

qualitative time constraints

ATime 56

A1

A2

A1

A2

[19,45]

[20,50]

51

t1

t2

t1

t2

46

t3

t4

Every task t can be represented by a timed interval [ ts a, te ] of two tasks ts and te such that ts < te

•

Allen [1983] distinguished 13 basic relations between timed intervals ( 6 + 6 converses + 1 equals relation) t

19 t3

[18,49] [18,55]

[19,55]

•

t

t4

t’

18

t t’

precedes

meets

17

with temporal info

t’

overlaps

t

t

t

t

t’

t’

t’

t’

finished by

contains

starts

equals

without temporal info



EASSS 2008

EASSS 2008

61

converting time interval ts

results for qualitative time

te

t

precedence

t’

t’s

precedes ts

synchro nization

meets

t’s


t’e

•

all 13 time interval relations can be represented by precedence (