From Active Objects to Autonomous Agents Submitted to IEEE Concurrency - Special Series on Actors & Agents Zahia Guessoum and Jean-Pierre Briot
Team Objects and Agents for Simulation and Information Systems (OASIS) Laboratoire d'Informatique de Paris 6 (LIP6), Case 169, 4 place Jussieu, 75252 PARIS Cedex 05 e-mail :
fZahia.Guessoum,
[email protected]
Abstract
This paper studies how to extend the concept of active objects into a structure of agents. It rst discusses the requirements for autonomous agents that are not covered by simple active objects. We propose then the extension of the single behavior of an active object into a set of behaviors with a meta-behavior scheduling their activities. To make a concrete proposal based on these ideas we describe how we extended a framework of active objects, named Actalk, into a generic multi-agent platform, named DIMA. We discuss how this extension has been implemented. We nally report on one application of DIMA to simulate economic models.
Keywords: active object, agent, implementation, meta-behavior, mod-
ularity, re-usability, simulation.
1 Introduction Object-oriented concurrent programming (OOCP) is the most appropriate and promising technology to implement agents. The concept of active object may be considered as the basic structure for building agents. Furthermore, the combination of the agent concept and the object paradigm leads to the notion of \agent-oriented programming" [11], which is the context of the present paper. The uniformity of communication mechanism of objects provides facilities to implement communicating agents and the concept of encapsulation of objects enables combination of various granularities of agents. Furthermore, the inheritance mechanism enables specialization and factorization of knowledge. An agent is easily implemented by an active object. Indeed the concept of an active object provides some degree of autonomy, in that it does not rely
on some external resources to be activated. Meanwhile, its behavior still remains procedural and only in reaction to message requests. More generally, we consider that, to be autonomous, agents must be able to perform a number of functions or activities without external intervention, over extended time periods. To achieve autonomy, several researchers have proposed to add to an active object some function to control the messages reception and processing by considering its internal state (see for example [2] [8]). The basic questions on how to build a bridge between: 1) the implementation and modeling requirements of multi-agent systems [3] [5] and 2) the implementation and modeling facilities and techniques provided by OOCP [6] are: - what is a generic structure to de ne the main features of an autonomous agent? - how to accommodate the highly-structured OOCP model into some relatively generic multi-agent implementation structure [6]? This paper is an attempt to provide answers to these two questions. It deals with 1) the modeling requirements of multi-agent systems by providing a generic and modular agent architecture and 2) the extension of the implementation and modeling facilities of OOCP. More concretely, we will describe how to extend a model of active object (named the Actalk framework) [2], towards a generic and modular agent architecture (named the DIMA platform). The paper is structured as follows. Section 2 presents brie y active objects, the framework Actalk, some limitations of active objects to represent agents and some requirements to build a generic agent structure. Section 3 describes the proposed generic agent architecture. Section 4 describes the implementation of this architecture with Actalk. Section 5 reports on the application of the proposed autonomous agents architecture to the modeling of economic agents evolution. Finally, we discuss the advantages of our architecture to design and implement multi-agent systems and we describe some future work.
2 Active Objects The concept of active object (also named actor) has been introduced by Carl Hewitt to describe a set of entities which cooperate and communicate by message passing. This concept brings the bene ts of object orientation (modularity, encapsulation, ...) to distributed environments and it provides object-oriented languages with some of the characteristics of open systems [1]. Based on these characteristics several models have been proposed (see for instance in [14]). 2
2.1 Actalk reception (receiveMessage:)
message (a Message/Invocation)
selection (nextMessage)
address (a MailBox) process actual computation (performMessage:)
activity (an Activity)
behavior (an ActiveObject)
address (an Address)
Figure 1: Components of an Actalk active object. Actalk [2] is a framework implementing various kinds of active object models [14] into a single programming environment based on Smalltalk. Asynchronism, a basic principle of active-object languages, is implemented by enqueuing the received messages into a mail box, thus dissociating message reception from its interpretation. In Actalk, an active object is composed of three components (see Figure 1): - an instance of class Address encapsulates the mail box of the active object. It de nes the way messages will be received and enqueued for later interpretation; - an instance of class Activity represents the internal activity of the active object. It provides autonomy to the actor. It owns a Smalltalk process and continuously removes messages from the mail box and launches their interpretation by the behavior component; - an instance of class ActiveObject represents the behavior of the active object, i.e. the way individual messages are interpreted. To build an active object with Actalk, one has to describe its behavior as a standard Smalltalk object. The active object using that behavior is created by sending the message active to the behavior. active
"Creates an active object by activating its behavior"
^self activity: self activityClass address: self addressClass
Methods activityClass and addressClass represent the default component classes for creating the activity and address components. Customizing the Actalk framework means de ning subclasses of the three component classes: Address, Activity and ActiveObject. This allows the 3
Activity
EnabledSetsActivity
GuardsActivity
AlwaysEnabledSetsActivity
ConcurrentActivity
SynchroConcurrentActivity
CountersActivity
EnabledSetsCountersActivity
GenericInvocationsCountersActivity
EnabledSetsGenericInvocationsCountersActivity
FullGenericInvocationsCountersActivity
Figure 2: A sample hierarchy of the activity/synchronization classes. user to de ne speci c models of active objects, e.g. various communication protocols as subclasses of class Address, various models of activity and synchronization as subclasses of class Activity (see Figure 2).
2.2 Limitations of Active Objects to Build a Generic Agent Structure
OOCP provides us with powerful foundations for modeling and implementing agents. However, these powerful and useful foundations do not really provide a generic agent structure. Active objects are monolithic and they have a procedural behavior. In spite of their communicating subjects appearance, active objects do not reason about their behavior, about their relations and about their interactions with other active objects. Also if an active object does not receive messages from other objects, it stays inactive. To cope with these limitations, several researchers have enriched the concept of active object to de ne a generic agent structure (see for example [8] and [11]). These are very interesting proposals, however they do not o er a generic agent structure which matches the whole spectrum of multi-agent systems requirements [6]. To make this more clear we will quickly summarize some needed agent properties [13] which are not provided by active objects: - An agent is not monolithic, it may have several behaviors: communication, perception, deliberation, etc. Each behavior can be procedural, but it can also include symbolic representations and substantial reasoning mechanisms. For example, the communication behavior may integrate knowledge about the agent acquaintances (other agents) and 4
a reasoning mechanism to reason about its interactions with these acquaintances. - An agent is an autonomous entity. It operates without direct intervention of humans or other agents. Therefore, it must have some control over its behaviors and its internal state [3]. To realize autonomy, each agent needs autonomous behaviors. For example, the communication behavior can reuse the active object communication mechanism but it must integrate a message interpreter to have some autonomy. The interpretation of a message relies on the agent state and the message contents. So, the same message may be interpreted di erently. The following example gives an interpretation rule which tests the mail box state and the priority of the received message to activate the suitable action. rule1 |Message message| conditions
self mailBox size < self threshold1. message priority >= self threshold2.
actions
self perform: message.
- An agent is a pro-active entity. It does not simply act in response to the received messages from the other agents. For example, an economic agent (see section 6) does not communicate with other agents, so it does not receive any message. This agent interacts with its environment (environment is made of non agent entities) and deliberates to choose the most appropriate action in the current context. So, it has two behaviors: perception and deliberation. The agent pro-activity allows to manage these two behaviors. The agent pro-activity is the self-control mechanism which manages the set of behaviors in accordance to the agent and its world states. We call world the union of the environment and the other agents. - An agent is a sociable entity. For example, agents are considered to form groups, but this notion of group is rarely provided by active objects. Moreover, agents may interact by speaking di erent languages. For example, in the agent communication languages such as KQML (Knowledge Query Manipulation Language) [12], the language of the message is indicated in an attribute of the received performative. This language can be an object-oriented language (Smalltalk, Java, ...) or an inter-change representation language (KIF, ...). - An agent is an adaptive entity. Its world (environment and other agents) is continuously evolving. So, some behaviors must be adaptive to make the agent capable of sustained performance. For example, 5
in the realized experiments (see section 6), results shows that it is dicult to improve the rm performance without predicting the competition performance. Furthermore, these results indicate that the use of case-based reasoning to predict the competition model changes does improve the agent performance. In the following section, we present a generic agent architecture that addresses such requirements.
3 A Generic Agent Architecture Supervision Module aThread ATN (States, Transitions)
anEngine Data
anEngine Data
Methods
... Behavior 1
anEngine Data
Methods
...
Methods
Data
... ...
Control
...
Behavior i
Behavior n
Figure 3: The proposed architecture. In attempt to de ne a generic architecture which addresses previous properties of an agent, we propose the extension of the single behavior of an active object into a set of behaviors. This architecture (see Figure 3) relies on a rst layer made up of interactive modules. These modules represent the di erent concurrent agent behaviors such as communicating, reasoning and perceiving. They can provide the agent with the properties described in Section 2.2. For example, the communication module manages the interaction between the agent and some other agents of the system. A higher level supervision module represents the agent meta-behavior. It allows the agent to manage the di erent behaviors.
3.1 Behaviors of an Agent
To model complex systems, agents need to combine cognitive abilities to reason about complex situations, and reactive abilities (stimulus-response). So, an agent may have two kinds of behaviors: reactive and cognitive behaviors.
6
3.1.1 Representation of Behaviors Each behavior encapsulates a set of data that de nes the behavior state and a set of methods (see Figure 3). The data can be updated by the behavior methods and/or by some asynchronous events. For example, the method scan of the perception behavior manages the interactions between the agent and its environment. It monitors sensors and translates sensed data to de ne a set of believes which can represent a model of the agent acquaintances and its environment. The methods can be either procedural (like standard object-oriented methods) or knowledge-based (a method that executes a knowledge-base). A behavior which only contains procedural methods is called a reactive behavior. Otherwise (it contains knowledge-based methods), it is called a cognitive behavior. Each behavior has its own engine which controls the activation of the various methods. A procedural behavior engine is a simple automaton. For example, the perception behavior engine of the agent rm de nes an in nite loop of activation of the single method scan of this behavior (see Figure 4. anEngine Domain Dependent Data
scan
scan state0
Perception Behavior
anEngine
Figure 4: Example of procedural behavior engine. Knowledge-based behavior engine is an inference engine. To implement the knowledge bases, we use the framework NeOpus [10] which realizes a neat integration of rule-based programming with Smalltalk-80. One of its prominent features is declarative speci cation of control with meta-rules. Meta-rules are rules which operate on so-called control objects. With each rule base there may be associated a meta-base which controls the ring of its rules. A number of meta-bases have been designed by NeOpus users to de ne standard types of control. We have adapted some of them to our needs by changing a few individual meta-rules to control the ring of the set of procedural and knowledge-based methods (an example is given in section 6).
3.1.2 Examples of Behaviors In this section, we give three examples of modules: the perception module (procedural behavior), the deliberation module (knowledge-based behavior) and the communication module (which can be either procedural or knowledge-based). 7
The perception behavior manages the interactions between the agent and its environment.
The deliberation behavior represents beliefs, intentions and knowl-
edge of the agent. It is responsible 1) for generating adequate responses to the messages transmitted by the communication module, or to the changes detected by the perception module, and 2) for achieving the agent goal(s). The communication behavior manages the interactions between the agent and the other agents of its group(s), no matter what machine they are running on. It de nes the mail box of the agent and the way the messages are received and enqueued for later interpretation. These three modules seem sucient to several application domains (see, for example, section 5). Agents often own a deliberation behavior, and a communication behavior and/or a perception behavior. Moreover, the use of a modular approach facilitates the integration of new modules such as a learning module.
3.2 The Agent Meta-Behavior
3.2.1 Issues
Several hybrid architectures were proposed (see [4] and [9]) to build agents out of two or more components which can be either reactive or cognitive. The reactive components are given some kind of precedence over the cognitive ones and the scheduling of these several components is static. Moreover, each component can be viewed as invoking a service in the underlying system which blocks the agent until the action has been completed. A key problem in such architectures is what kind of control can be used to manage the interactions between these fundamentally di erent components: reactive and cognitive components. Perception
Decision
Figure 5: A static scheduler Let's consider a simple economic agent with two behaviors: a perception and a decision components. Figure 5 gives an example of static scheduling that is implicitly used in most existing architectures (see for example [4]). The agent activity may be summarized within a \perception and then decision" loop. 8
To underline this problem of control, we give the following scenario: At time t, Firm 1 has a model of its competitive Firm 2 which has a greater price and a lower quality. Firm 1 activates its decision process. Parallel to this process, Firm 2 decreases the unit price and improves the quality of its product. If Firm 1 does not deal rapidly with these modi cations, its product would not be consumed. To avoid this problem, a solution is to focus control attention without prescribing a static scheduling. Each agent has to adapt rapidly its scheduling to changes in its world (the other agents and the environment). So, the agent scheduler cannot be static as proposed in most existing architectures.
3.2.2 Our solution Many systems have emphasized the need for explicit and separate representation of control or the re exive aspect of meta-level architectures. Following this tradition of explicit and separate representation of control, we propose a meta-behavior in our agent architecture. This meta-behavior gives each agent the ability to make appropriate decisions about control or to adapt its behaviors over time to new circumstances. It provides the agent with a self-control mechanism to dynamically schedule its behaviors in accordance to its internal state and its world state. This solution has the following foundations: - behaviors are explicitly and dynamically scheduled by the meta-behavior, - behaviors are explicitly interrupted to allow the meta-behavior to adapt its scheduling according to new events. Procedural behaviors are provided with a time granularity of a single procedural method ring and knowledge-based behaviors are provided with a time granularity of a single rule ring. In other words, at the knowledge-based methods level, scheduling is performed after each rule ring. This solution is acceptable in many industrial applications [7].
3.2.3 Meta-Behavior Representation The proposed meta-behavior relies on two fundamental notions: states and transitions which naturally build up an Augmented Transition Network (ATN). Figure 6 gives an example of ATN that may describe an economic agent meta-behavior. States represent decision points. They are used to choose the next transition beyond the associated transitions. Each transition is labeled if condition then action. It represents a step of the global scheduling of the agent and links an input state with an output state. The conditions of transitions test the occurrence of an asynchronous event (urgent message reception, new data, ...). These asynchronous events 9
if true
if true
then activatePerception
then activateDecision if
State1
State2
noImportantChange and suspendedDecision
State3
then activateDecision if
importantChange or decisionTerminated
then activatePerception
Figure 6: A rst example of ATN of an economic agent often operate changes on the behaviors data. For example, noImportantChange tests if a new data has occurred and if it provides an important change according to the previous data. The actions of transitions allow the management of the rst layer behaviors (activate reasoning, terminate reasoning, activate communication, activate perception, ...), the meta-behavior has to choose which behavior it should activate. if if if
true
true
then activateDecision
then activatePerception State1
State2
noImportantChange and suspendedDecision
then activateDecision
State3
if
importantChange or decisionTerminated
then activatePerception if stopCondition then kill
if stopCondition then kill
if stopCondition then kill
FinalState
Figure 7: A second example of ATN of an economic agent Moreover, the meta-behavior may evaluate its behaviors and stop its behaviors when needed. So, each agent has a condition stopCondition and an action kill (the whole agent). Figure 7 gives an example of an ATN which uses stopCondition and kill. The meta-behavior has also a data aListofBehaviors which can be updated. For example, it can remove a behavior or add a new one. At each state, the ATN-based meta-behavior evaluates the conditions of transitions (representing new events) to select the most appropriate behavior. When these conditions are veri ed, the actions of the associated transition are executed and then the agent state is modi ed. The meta-behavior represents the agent self-control mechanism. This self-control provides the agent with some kind of control over its behaviors and its internal state. It de nes the agent pro-activity which is not restricted to message reception/sending. Therefore, it makes the agent autonomous by allowing it to operate without direct intervention of humans or other agents. 10
Moreover, it makes the agent adaptive by allowing it to deal rapidly with new events.
4 Implementation of DIMA with Actalk In DIMA, a multi-agent system is a set of agents and also possibly a set of objects representing the agents environment. To implement a multi-agent system, one has to implement the environment, which is a collection of simple Smalltalk objects, and then to implement the agents by customizing DIMA. The use of an object-oriented language brings the bene t of the inheritance mechanism. Therefore, customizing DIMA means using or subclassing the hierarchies of classes (see Figure 8). This allows to de ne: - the agent behaviors, - the agent meta-behavior, - the agent engine.
Figure 8: Classes describing the agent components. 11
4.1 The Agent Meta-Behavior
In DIMA, we decouple the agent meta-behavior (described by an ATN) and the agent engine (which interprets the ATN). As for expert systems which decouple knowledge bases with the inference engine, this achieves declarativity and modularity. As the agent meta-behavior represents the analog of the behavior of an active object, it is implemented as class Meta-Behavior, de ned as a subclass of Actalk class ActiveObject. The agent engine represents the activity of the agent and thus is implemented by class AgentEngine, de ned as a subclass of Actalk class Activity (See Figure 8.) The class Meta-Behavior has an attribute anATN which de nes the metabehavior, and an attribute aListofBehaviors which de nes the collection of behaviors of the de ned agent. This class implements: - conditions and actions of the ATN transitions, - one or several methods for creating agents. The main steps to describe an agent are the followings: 1. Determination of the agent behaviors. 2. Implementation of the classes describing its behaviors by sub-classing or using existing classes (see Figure 8). 3. Implementation of the agent ATN by instantiating the class ATN. 4. Creation of the agent by using an appropriate method de ned in the class Meta-Behavior. Example of method: agent := Meta-Behavior newAgent: aSymbol listofBehaviors: (OrderedCollection with: aDeliberationBehavior with: aPerceptionBehavior) atn: anATN.
5. Activation of this agent engine (agent
4.2 Agent Engine
resume).
In Actalk, the class Activity represents the internal activity of the active object. The instance method body, used by createProcess, de nes the basic loop of the active object which serially processes messages of the mail box. !Activity methodsFor: 'activity setting'!
body
[true] WhileTrue: [self acceptNextMessage]
createProcess
[self body] newProcess
12
To implement the agent engine, we have sub-classed the class Activity of Actalk to de ne the class AgentEngine. In the latter, the method body has been rede ned: !AgentEngine methodsFor: 'activity setting'!
body
self atnInterpreter
!AgentEngine methodsFor: 'atn'!
atnInterpreter
|atn state| atn := self metaBehavior atn. state := atn initialState. state = atn finalState whileFalse:[state := atn transitionAt: state]
Note that, in the agent-oriented programming [11], the cycle of knowledge inference implements the cycle of message acceptance. So, the agent activity is limited to the message reception and sending. In DIMA, the agent activity is represented by the ATN interpreter and is made explicit to programmer.
5 Advantage of the Proposed Agent Architecture Thanks to its modularity, the proposed architecture helps in decomposing the arbitrarily complex behavior of an agent into a collection of small specialized behaviors, and supervising these various behaviors by a meta-behavior. The advantages of this architecture match the agent properties discussed in section 2.2. They may be stated as following: - multi-granularity: Agents of various granularities (size, internal behaviors, knowledge) may be expressed with the proposed architecture. This property is very important in the design of complex systems. It goes further than the classical dichotomy between reactive and cognitive agents. - dynamicity and openness: Agents may be dynamically created and/or killed. They may integrate new behaviors and change their acquaintances along the information they receive and/or they perceive. New behaviors may be created and integrated by the agent metabehavior along the needs for new skills. - re ection: Our architecture implements an agent-based model of re ection in which each agent has its own meta-behavior which governs its various behaviors, e.g. in order to make appropriate decisions about control or to adapt its behaviors over time to new circumstances. 13
- heterogeneity: Heterogeneity is a very important issue in multiagent systems. However, most existing systems do not accept heterogeneous agents such as agents implemented with or speaking di erent languages. The proposed architecture is mainly characterized by: 1) the separation of the agent communication behavior from the deliberation behavior and 2) each communication behavior has a messageinterpreter to translate the received-message language to an internal agent language. These two characteristics allow to have more than one language in the same multi-agent system.
6 Experiments To validate the operational platform (DIMA), we have developed several applications. In this section, we report on some application to simulate the economic models. The economic model that we chose is the result of an extensive research which was conducted on a representative sample of the manufacturing rms. The database is collected by the French National Bank (Banque de France). In this application, we consider a set of economic agents in competition with each other in a market. These agents have two behaviors: perception and deliberation. The meta-behavior was described in Section 3.2 (see Figure 7).
6.1 The Firm Perception Behavior
The perception behavior allows the rm to observe the market and to build a competition model. The observable data are the ones shown in the market such as the data representing the performances of the various rms. To represent this behavior, we have implemented a class : FirmPerceptionBehavior (sub-class of PerceptionBehavior). This behavior has only a method (called scan) which aims to build a competition model by observing data changes in the market. scan ``yCollection is a collection of performances of the other firms'' yCollection := self scanMarket. self updateMemory. self buildCompetitionModel.
Therefore, an elementary competition model can de ne data such as the best, average and worst performances. buildCompetitionModel ymax := yCollection max. yave := yCollection average. ymin := yCollection min.
14
This class also implements a set of conditions that are used in the ATN. For example, the method noImportantChange (see Figure 7) means that the new observed collection of data is very similar to the previous one. noImportantChange ^self testSimilarityOfData
This similarity is de ned by the following expression :
X n
=1
j
Wi Yt;i
? Y ?1 j= t
;i
Xj n
Yt;i
=1
? Y ?1 j < t
;i
(1)
i
i
where is a constant ( 0.05) and W are weight associated to each Y performance variable. These weights was de ned by the expert (economist). Note that: W = 1: (2) i
X
i
n
i
=1
i
6.2 The Firm Deliberation Behavior
A rm is de ned by the following main properties: - The state variables (x vector) represent the di erent types of resources (funds, people, equipment, ...) owned by the rm. - The y variables represent the performances of the rm. They are directly in uenced by the x vector. - Finally, a rm is characterized by the strategy it follows to allocate its resources. In our model, a strategy is an order of priority for modi cation of the x vector. For instance, the cost strategy concentrates on the x vector variables that are related to the production resources. x Strategy Choice and Application
Internal Parameters Modification
Modification of Parameters
y
Compute v
Send the New Parameters to Transaction Zone
Competition Model
Figure 9: Firm deliberation behavior. Figure 9 describes the rm deliberation behavior methods, each box describing a method. For example, \ComputeV" is a procedural method. On the other hand, \StrategyChoiceAndApplication"is a knowledge-based method which chooses a strategy. This method is implemented as a simple knowledge base with NeOpus [10]. In the following, we give an example of rule. 15
!FirmRuleBase methodsFor: 'init'!
chooseStrategy1 conditions
|FirmDeliberationBehavior b| b v decreases.
actions
b applyStrategy1.
So, class FirmDeliberationBehavior is subclass of KnowledgeBasedBehavior. As it was underlined in section 3.1, the engine of this behavior is the inference engine of the system that we use to implement the knowledge base. This system is based on a set of meta-rules. In the following, we give two examples of meta-rules of this behavior. !FirmMetaBase methodsFor: 'firm engine'!
initNoObjects | Evaluator e | conditions
e status = #init. e context exists not.
actions
self computeV. e status: #modifyInternalParameters. e modified.
!FirmMetaBase methodsFor: 'firm engine'!
strategyChoiceEnd | Evaluator e| conditions
e status = #strategyChoice. e reussi.
actions
e strategyChoice e status: #modifyParameters. e modified.
6.3 The Simulation Experiments
The aim of these experiments is to underline some properties of the proposed agent architecture.
6.3.1 Population of Agents
The rst property we try to underline is that agents can be added dynamically and can also leave the system. So, an agent meta-behavior (see gure 7) can have one or more transitions with: - The condition stopCondition which is related to the application domain. For an economic agent, it is de ned by the expression: capital
