Expressing Evolutionary Computation, Genetic Programming, Artif icial Life, Autonomous Agents and DNA-Based Computing in $-Calculus - Revised Version1 Eugene Eberbach Jodrey School of Computer Science Acadia University Wolfville, NS B0P 1X0, Canada
[email protected] Abstract- Genetic programming, autonomous agents, artif icial life and evolutionary computation share many common ideas. They generally investigate distributed complex processes, perhaps with the ability to interact. It seems to be natural to study their behavior using process algebras, which were designed to handle distributed interactive systems. $-calculus is a higher-order polyadic process algebra for resource bounded computation. It has been designed to handle autonomous agents, evolutionary computing, neural nets, expert systems, machine learning, and distributed interactive AI systems, in general. $-calculus has built-in cost-optimization mechanism allowing to deal with nondeterminism, incomplete and uncertain information. In this paper, we express in $-calculus several subareas of evolutionary computation, including genetic programming, artif icial life, autonomous agents and DNA-based computing.
1 Introduction In this paper we try to demonstrate that not accidentally genetic algorithms, genetic programming, classif ier systems, evolution strategies, evolutionary programming, DNA-based computing, evolvable hardware, artif icial life, autonomous agents are combined in many conferences. All these subdisciplines share a lot in common. Evolution based adaptation and numerical f itness measure belong to the two most important shared features. The above motivation have been expressed here in a formal way, presenting an integrating approach to study all these subdisciplines in one framework, whose main features are just modif ications (not necessarily based on evolution) and cost metrics (including f itness). The approach aims at developing a general model for distributed systems with intelligence. It has been designed to specify, investigate, and implement systems exhibiting meta-computation (e.g., selfmodif ication) and optimization, including expert systems, neural networks, evolutionary computation, artif icial life, adaptive autonomous agents, plan generation, fault-tolerant systems, computer viruses, machine learning, and evolvable hardware. The $-calculus (read: cost calculus) [9, 10, 11, 12, 13] is a higher-order polyadic process algebra with a quantita1 research partially supported by a grant from NSERC No.OGP0046501, a preliminary version of this paper has appeared in Late-Breaking Papers of Genetic Programming GP’98 Conference, Madison, Wisconsin
tive aspect (cost) which naturally expresses adaptability (optimization) and self-modif ication for interactive distributed systems. Agents interact and infer by send, receive and mutation primitives. It differs from other models by including cost (time, resources, etc.) as a basic component expressing the optimization behavior typical of many AI systems, and in particular, resource bounded computation. Resource bounded computation attempts to f ind the best answer possible given operational constraints. The approach is known under a variety of names, including f lexible computation, anytime algorithms, imprecise-computation, or designto-time scheduling [6, 17, 27]. Anytime algorithms together with closely related decision-theoretic metareasoning are two new promising techniques and their application to general decision-making architectures has not yet been investigated in any depth [27]. $-calculus formalizes resource bounded computation using process algebras. It is related to other well established models for sequential and parallel computation, including calculus [5] and -calculus [24, 25], which have been extended by utilities/costs allowing to capture bounded rationality of computing agents. $-calculus is based on true concurrency semantics, and allows both for process migration and link mobility (needed to express reconf igurable topologies), and incorporates higher-order features (needed for automatic program synthesis). It has been designed specially to handle autonomous agents, evolutionary computing, neural nets, expert systems, machine learning, and distributed interactive AI systems, in general. The $-calculus leads to a new programming paradigm: cost languages, and a new class of computer architectures: cost-driven machines [8]. Agents interact and infer in $-calculus by send, receive and mutation primitives. $-calculus has built-in costoptimization mechanism allowing to deal with nondeterminism, incomplete and uncertain information. It allows a uniform mechanism to express control (by numerical feedback), optimization, learning, adaptation, uncertainty, incompleteness or vagueness (costs interpreted as probabilities, a fuzzyset membership function), classif ication errors in neural networks, f itness from genetic algorithms, the minimum description length, entropy, level of interestingness, energy level, distance, time, complexity, etc. To avoid combinatorial explosion, on-line optimization often considers a restricted subset of agents, and only a few steps of execution. In $-calculus, self-modif ication and cost-optimization are the driving force for problem solving, control and automatic pro-
gram synthesis. The innovative aspect of $-calculus is that it integrates neural networks, genetic programming/algorithms, symbolic rule-based expert systems, logic, imperative and object-oriented programming in a common framework. In this paper, we concentrate on evolutionary computation applications of $-calculus. We describe basic syntax and operational cost semantics of the calculus needed for understanding the remaining sections. Crisp, probabilistic and fuzzy-logic cost functions are def ined for strongly and weakly congruent cost expressions. A meta-level control (a simple modifying algorithm) is used to evolve a modif ied algorithm in the solution of so called k -optimization problems. We demonstrate how to express in $-calculus several subareas of evolutionary computation, including genetic programming, artif icial life, autonomous agents and DNA-based computing. We believe that this should be useful for the future development of the evolutionary computation area.
2 Cost Expressions: Syntax and Description Let T be a set of terminals (data) corresponding to the inputs (constants, variables, data structures, or suppressed functions). They are called terminals because they terminate (suppress) evaluation of expressions. Let F be a set of functions (code), which may be standard arithmetic operations, programming operations, mathematical/logical functions, user or automatically def ined functions. Functions are executed by evaluation. Terminals are data for functions. Functions are dynamic, they take input terminals, and modify them, producing as the output new terminals. Let M be a set of modif ications (meta-code) taking as input and returning as output suppressed functions, or function def initions. Modif ications and costs are unique and important features of the $-calculus. Modif ications are an engine searching for the best algorithm (minimal costs) to achieve the desired goal. In other words, terminals can be considered zeroorder functions, regular functions are of the f irst order, and modif ications - higher-order ( 2) functions. The uniform approach to data, code and meta-code is similar to Lisp, and together with cost optimization, is believed to be crucial to our approach. Throughout the paper we will use a pref ix notation, i.e., we will write data, functions, and modif ications uniformly as (f ~x), where f is a function/terminal/modification name (label) and ~x = (x1 ; x2 ; :::) is a vector of its parameters (possibly countably inf inite). Sometimes if it is understable from the context, we will omit parameters, and we will write simply f . Also if it understable we will call data, functions, and modif ications, all simply functions. Let a; b; c; ::: be names over sets T , F and M , and each with an arity - an integer 0 (representing number of parameters) for each terminal, function and modif ication. Let ; k ; [[ ; t; $; :; !; ; 7!; := be predef ined (reserved) names. We will assume also that basic arithmetic and relational operators are predef ined and available to be used, e.g. (+ 1 2),
(< x 4) etc.
Let P, Q, R, ... range over the $-expressions P (read: cost expressions, also can be called agents, programs, processes, objects) and let X, Y, Z, ... be process variables. We will distinguish simple (or atomic, contract) $-expressions, executed (evaluated, reduced) in one indivisible step, or considered to be indivisible because it is more convenient from the point of a user (like atomic transactions in databases). $-expressions return other cost expressions as the result of their evaluation (in particular, they may return nothing after successful execution, or they may block after a failure in execution). Then the syntax of $-expressions is the smallest set P , which includes the following kinds of cost expressions (assuming that P; Q; Pi are arbitrary cost expressions), i.e., compund (interruptible) $-expressions:
P; Q; Pi
::=
( i2I Pi ) ( k i2I Pi ) ([[ i2I Pi ) (ti2I ( i Pi )) (f Q~ )
sequential composition j parallel composition j cost choice j general choice j call of (user) def ined $-expr. (application) j (:= (f X~ ) P ) recursive def inition (abstraction) Simple (atomic, contract) $-expressions: ::= ($ P ) cost j (! (a Q~ )) send j ( (a X~ )) receive j (7! (a Q~ )) mutation j (a Q~ ) call of (user) def. simple $-expr. j (:a Q~ ) negation of (user) def ined simple $-expr. Cost, negation, choices, compositions, send, receive, mutation, primitive arithmetic, logic and relational functions are examples of built-in functions. Other functions and data structures can be specif ied using def inition. In $-calculus everything is a cost expression: agents, environment, communication/interaction links, inference engines, modif ied structures, data, code, and meta-code. Here I is a possibly countably inf inite indexing set. In the case I = ; we will write the empty parallel composition, general and cost choices as ?, and the empty sequential composition as ". Sequential composition is used when $-expressions should be evaluated in a textual order. Parallel composition is used when expressions should run in parallel, Cost choice should be used when we are interested in optimization, i.e. it selects the cheapest alternative according to some cost metrics. Call and def inition allows to encapsulate expressions in a more complex form (like procedure or function def initions in programming languages). In particular, they allow to specify recursive or iterative repetition of $-expressions. Simple cost expressions are executed in one atomic step, and they can be either negated or not. Cost function is used for optimization and adaptation, and currently consists of
crisp, probabilistic, and fuzzy cost functions. User is free to def ine his/her own cost metrics. Send, mutation and receive are used for handshaking message-passing communication, but also for inferencing. The indexing set I must be inf inite to express formalisms with richer behavior than Turing machines: including cellular automata [4], interaction machines [31, 32, 33], neural networks, and random automata networks [14]. This expressive power is needed to express construction universality, selfreproduction, and evolution problems. [13] shows $-calculus simulates -calculus, -calculus, interaction machines, cellular automata, neural networks and random automata networks. Example 1 send and receive are similar to polyadic input and output pref ixes from -calculus. The main difference is that send and receive always have to occur in pairs, i.e. they are blocked if executed separately (in -calculus, this is equivalent to use the restriction operator enforcing simultaneous communication of input and output pref ixes). Another difference is that cost can be used to f ind which pairs will communicate. We will show a few examples of successful send/receive:
( k ( (! (a 2)) Q)( ( (a x)) P ) = ( k (Q P f2=xg) , and ( k (! a) (! a) (( ( a) b) ( ( a) c))) = ( k b c),
Example 2 Mutation is form of communication which can return various cost expressions (depending on the choice of the mutation function/channel). It is equivalent to send through the noisy channel. In one atomic step an expression is mutated (evaluated), sent, and received by corresponding receive. Send can be considered as a special case of mutation through an ideal identity channel. For example, let’s consider mutation a f lopping 1 to 0:
( k ( (7! (a 1)) Q)( ( ( k Q P f0=xg)
(a x)) P )) =
Mutation can simulate many other operators. For instance, a classical one-point crossover from evolutionary computation exchanging Q1 and Q2 in expressions P1 and P2 can be simulated by ( k (7! (a Q1 Q2)) ( ( (a x y) ( k P1 P2 ))) = ( k P1 fQ2=xg P2 fQ1=yg), where mutation channel a swaps its arguments Q1 and Q2 . This can be considered as another argument to the discussion which operator: crossover or mutation (macromutation) is more essential for evolutionary computation.
3 Operational Cost Semantics and Behavioral Equivalence of $-Expressions: Strong and Observation Congruence The operational semantics of $-expressions has been def ined in a conventional way using a Labeled Transition System (LTS), which leads to the notion of a derivation tree recording the successive transitions or actions which may be performed
by given process. We will skip the mathematical details of it. The main idea is that instead of seeking the smallest derivation tree, $-calculus looks for the set of transitions with minimal cost. If all actions have the same cost, then the derivation tree found by the $-calculus will be identical to the derivation tree found using the smallest action relation. Operational cost semantics describes which atomic multisets of actions can be applied to current cost expressions leading to new cost expressions. Example 3 To illustrate the minimal cost action relation semantics, consider the following derivation tree:
a ([[ " ( b ")) ([[ ( a ") ( b ")) ?! We assume that ($ ( a ")) = 7 ($ ( b ")) = 10, thus action a has been selected. We also assume that cost of the silent action is zero ($ ") = 0, i.e., the derivation of the tree has been terminated, because to choose the silent action " is cheaper than to execute ( b ").
We def ine strong and observation congruence in a conventional way for process algebras. Strong and weak (observation) congruence equate processes which possess the same transitions (counting or not invisible action ") and deadlock in the same way. They differ in the approach to invisible action " and requirement for matching of f irst actions. Different types of equivalences are important, because equivalent processes should have the same cost, and preferred process should have a smaller cost. This follows directly from the von Neumann/Morgenstern utility theory.
4 Composability of Performance Measure: Cost Functions Cost (utility) functions represent a uniform criterion of adaptation (control). Cost functions may have many possible interpretations and formulas. For this reason we do not assign one particular def inition to be used within the calculus, Nevertheless, a few typical examples of costs include functions representing time, money, f itness, error, probability, possibility, belief, complexity, distance, energy, entropy, the number of resources, etc. For instance, the costs representing probabilities of cost expressions lead to the probabilistic $calculus, and the costs interpreted as the fuzzy set characteristic function - lead to the fuzzy-logic $-calculus. Costs can be explicit or implicit (endogenous like in Echo system [16]). Explicit costs can be understood as an abstraction of the measure how well the system (agent, program, process) performs in the environment. In implicit costs, the environment provides the direct feedback, fulf illing as the result a similar function as explicit costs. Costs expressing performance of multiple agents or evaluating several factors can be combined in one cost to express multiobjective optimization (using, for instance the weighted sum of component costs), or they can be kept separately as a vector of component costs (this can be useful, for instance, for autonomous adaptive agents or co-evolutionary systems).
In multimodal optimization we are looking for several or all optima for a given cost function. Costs can be both positive, and negative (i.e., prof its, payoffs), or any domain where the linear order is def ined. To compute the cost, the most important is the compositionality principle: “the cost of the problem consisting of subproblems is the function of costs of subproblems”. Sometimes this function is diff icult to f ind and easier is to assign cost to the program as the whole. This is the biggest difference compared to f itness function, which is def ined to the whole gene, however, composability is consistent with the utility theory or dynamic programming. In the utility theory, such desirable property of the utility function is called separability. $-calculus has predef ined 3 typical examples of cost functions: standard crisp, probabilistic and fuzzy. Users are free to def ine their own cost functions. We will present here only for illustration a crisp cost function. Let P be the set of $-expressions. The user is responsible for specif ication of costs of simple terminals, functions, and modif ications (costs of simple cost expressions - elements from T , F and M ) by cost action prof iling (e.g., using empirical methods). The domain of the cost function is R1 (nonnegative real numbers with added inf inity), i.e. $ : P ! R1 . Let v : A" ! R1 be user def ined costs of simple cost expressions, including a silent expression. They can be context dependent. In particular, cost of " may depend which cost expression is made invisible by ". Note that the value of the cost function (or its estimate) can change after each step (evaluation of a simple cost expression). (A standard crisp cost function)
($ ?) = +1 for observation congruence 0 2. ($ ") = (v ") for strong congruence 3. ($ ) = c + (v ), where c = 0 does not block +1 blocks ($ (: )) = c1 + (v (: )) 1.
($ (ti2I ( i Pi ))) = i2I di (($ i ) + ($ Pi0 )) where di is the probability of choice of the i-th branch [ i2I Pi )) = ($ j ) + ($ ( [ [ i2I;i6=j Pi Pj0 )) = 5. ($ ( [ (mini2I (($ Pi ) + gi )), where gi is the cost history of the i-th component, initially 0, and after execution of j i = j and history on gi = ggi + ($ i ) for for i 6= j or history off i
4.
Pi )) = ($f j gj2J )+($( k i2I ?J;j2J Pi Pj0 )) = (minK I;K 6=; ($( k k2K Pk ))), where ($ f j gj2J ) = j2J cj ($ j ), and ck = 1 for ($ k ) = (maxj2J ($ j )), 0 cj 1 for remaining j 6= k.
6. ($ k i2I
7.
~ X~ g) where (:= (f X~ ) P ). ($ (f Q~ )) = ($ P fQ=
Cost functions have been def ined in such a way that equivalent (strongly or observation congruent) $-expressions have the same costs. In the cost expressions consisting of general and cost choices, parallel and sequential compositions, and recursive def initions, cost optimization is associated with cost choice and parallel composition operators. In the cost choice the cheapest alternative is attempted to be selected, and cost of each alternative is calculated as the cost of a current action and the cost of remaining actions. The same applies to parallel composition: it looks for the maximal subset of actions which are cheaper to execute than to execute them sequentially. The decision is based on costs of current and remaining actions. However, the cost of remaining actions can be either to expensive to compute, not available, or can change in time. Then instead of precise values of remaining actions, we use their estimates. The same applies to the situation that costs of some actions are not available, because they are executed by another agent. The question is, can we f ind an optimal solution, in spite of approximate cost values and in spite of wrong decisions? The answer is yes, but it depends whether we count (history off) or disregard (history on) costs of mistakes, i.e., what we consider to be optimal. If to disregard costs of mistakes, history is on, and costs [ will be explored. are not overestimated the cheapest path in [ If to count the cost of wrong choices, which is equivalent that history is off, the only way to guarantee that the optimal solution is found is in fact to know in advance which branch is optimal. The above means that if history is off, the optimization will be local only unless estimates do not cause errors.
5 Meta-Level Control: A Simple Modifying Algorithm The basic control structure of the $-calculus, so called simple modifying algorithm is very simple, but powerful. It is used almost without changes in expert systems, neural networks, or genetic programming. The meta-level control of the $-calculus can, however, take the form of more complex algorithms (as a basic structure of an evolutionary algorithm, applicable practically without changes to genetic programming, genetic algorithms, evolution strategies, evolutionary programming, and classif ier systems, may lead to a more complex form like the cultural algorithm, or the genocop algorithm [23]). A simple modifying algorithm is the following cost expression with modif ications (higher-order functions) loop, sel, exam, exec, !, ,7!, and user specif ied $, init and term, working upon population of programs P (modif ied algorithms) constructed of terminals and built-in functions , k , [[ , t, !, , 7!, :, and := (used for user def ined functions):
( (init T F ) /* initialize P from fT; F g */ (exec P ) /* run P and f ind its cost */ (loop P ) /* basic cycle: select, examine, ) execute */ (:= (loop P ) /* loop recursive def inition*/ (t ( (:term P ) /* termination not (sel P ) (exam P ) (exec P ) )
(loop P )) (term P ))
satisf ied */ /* select: select subexpressions, perform modif ications (using mostly 7!, !, ) */ /* examine: optimize P using structural congruence, remove [ by nondeterminism in [ taking the cheapest branches */ /* execute: run P and f ind its cost */ /* return back to loop */ /* terminate P */
The $-calculus expresses adaptation as an optimization process. Global behavior is optimized by performing local optimization of components. “Fuzzy” concepts like emerging intelligence are given a mathematical foundation. System design is reduced to f inding an appropriate objective cost function, and the choice of optimization parameters, methods, algorithms. Because local optimizations are performed, the complexity of global (perhaps, intractable) optimization is transformed to simple (feasible) local optimizations. The most challenging and interesting question is which behaviors (local optimizations) should be performed by components and how to orchestrate their co-operation, to achieve the desired effect. This is resolved by masking parts of the problem-space from a given agent in the solution of so called k -optimization problems (implementing loop of the simple modifying algorithm). This provides an approach for local and global optimization in time and space. Parts of $expressions (note, in $-calculus everything is a $-expression: agents, their behavior, data, interactions, environment, cost functions) are replaced by silent $-expressions, which mask part of the problem space (either spatially or temporally). For observation congruence, silent $-expressions are invisible with a neutral cost 0. They do not participate directly in optimization, however, they do participate indirectly and inf luence the results. For strong congruence, silent actions (representing incomplete knowledge of an agent) are invisible, but their cost is not zero. Depending on the cost estimation, global optimization may or may not be successful.
6 Expressing EC, GP, ALife, Agents and DNA Computing 6.1 Evolutionary Computation and Genetic Programming An arbitrary evolutionary algorithm [23] (which subsumes genetic algorithms, evolutionary programming, evolution strategies, and genetic programming) is very similar to a simple modifying algorithm: ( (init T F ) /* initialize population P of chromosomes */ (eval P ) /* run P and f ind its cost/f itness */ (loop P ) /* basic cycle: select, ) alter, evaluate */ (:= (loop P ) /* loop recursive def inition*/ (t ( (:term_cond P ) /* termination condition not satisf ied */ (sel P ) /* select: select surviving individuals based on f itness */ (alter P ) /* alter: apply crossover and mutation */ (eval P ) /* evaluate: run P and f ind its f itness */ (loop P )) /* return back to loop */ (term_cond P )) /* the termination ) condition is satisf ied */ A Genetic Programming [18] procedure is the following simple modifying algorithm with modif ications loop, sel, exam, 7!, and user specif ied $ and term, working upon population of imperative/functional (sequential and deterministic) programs P constructed of user selected terminals and functions. Terminals are limited to atoms: constants, variables, and zero-argument functions. Terminals and functions take the form of user def ined simple $-expressions. Additionally, user selected functions may include analogues of (PROGN from Lisp), t (IF-THEN-ELSE), and := (ADFs). Automatically Def ined Functions (ADFs) take the ~ ) P ), where, as the standard, crossover form: (:= (f X (and other modif ications - secondary GP operations) are limited to body P , return values are f ixed, and no recursive def initions are allowed. The cost function is a standardized GP f itness, calculated from raw f itness provided by the user and specifying, how well the program as the whole solves the problem. The best program is with minimal standardized f itness (nonnegative reals). init generates randomly population of programs P , The select phase sel consists of the selection based on cost/f itness, and mutation 7! and reproduction modif ications. The examine phase alter is limited to editing modif ication simplifying programs using laws of programming. The execute phase eval consists of running programs and calculating their cost/f itness.
Compared to classical GP, $-calculus introduces standardization of f itness/cost functions (and most imortant - its composability), standardization of other functions (e.g., choices, compositions, recursion), and allows to evolve parallel, recursive, nondeterministic, and distributed interactive programs. 6.2 Artif icial Life and Cellular Space Models Artif icial Life studies biological phenomena by attempting to reproduce them within an alternate media: software (virtual life), wetware (alternative life) or hardware (synthetic life) [19]. Typical representatives of each category are cellular automata, biomolecular engineering, and cellular computers each operating in its own cellular environment. The research on self-replicating systems in cellular space model is important because it considers the possibility of self-reproduction, self-repair, and cooperation of multiple cells (perhaps infinite many of them). Some systems [21] allow for self-replication (creation of a copy of the original structure, which appears isolated, and possibly rotated), and some for universal computation (in the sense of universal Turing machine), and some for universal construction (a machine can construct whatever machine’s description is given as input, including itself). Categories of cellular space models include cellular automata, non-uniform cellular automata [28] (not identical transitions rules), and models with complex automata networks (not identical transitions rules and graph is not regular). We will concentrate on cellular automata. Cellular automata [4], originally due to von Neumann and Ulam are a classical example of ALife [19]. Formally, a cellular automaton [14] is a pair (D; fM g) consisting of a cellular space (D; fQg) (countably inf inite, locally-f inite regular (each node has the same degree) directed graph D with states Q assigned to each node) and common f inite-state machine M with input alphabet = Qd , Qd = Q ::: Q (d times) and local transition functions
:Q!Q The global evolution (dynamics) of a cellular automaton is best viewed as a discrete dynamical system (self-map) T : C C , where C = i Q is a set of conf igurations (total states)
T (x)i = (xi ; xi1 ; :::; xid )
A cellular automaton operates locally as follows. A copy of a common f inite state machine M occupies each vertex of a regular graph (cellular space) which is then called a cell. Synchronously, each copy of M looks up its input in the states x1 ; :::; xd of its neighboring cells and its own state x0 , and then changes its state according to its local dynamics . The cellular automaton performs its calculation by repeating these atomic local rules any (possibly very large) number of times for all sites resulting in emergent behavior (global dynamics) of the whole system. In $-calculus we simulate a cellular automaton CA as follows. We def ine an i-th copy of f inite-state machine M
occupying the i-th node as (:= (Mi xi xi1 :::xid ) ( k (7! (ai ( xi xi1 :::xid ) ( xi1 xi11 :::xi1d ) ::: ( xid xid1 :::xidd )) ( ( (ai xi xi1 :::xid )) (Mi xi xi1 :::xid )) and CA as
( k i (Mi xi xi1 ::: xid )) d neighbors calculate
new values of Mi and all its xi ; xi1 ; :::; xid and pass them by mutate and receive pair to a new call of Mi . If the cost function is def ined in such a way
that parallel actions are cheaper than sequential ones, then the parallel composition will execute all transitions for all copies of M simultaneously. This means than $-calculus can simulate an arbitrary cellular automaton. Self-replication, universal computation and universal construction can be studied in the context of $-calculus with a slight difference. The notions of universal computation and universal construction should be upgraded, because $calculus allows to express models having richer behavior than Turing machine.
6.3 Adaptive Autonomous Agents and Evolutionary Robotics Adaptive autonomous agents are systems that inhibit a dynamic, unpredictable environment in which they try to satisfy a set of time-dependent goals or motivations. Agents are adaptive (intelligent) if they improve their competence at dealing with these goals based on experience. Dissimilar to conventional AI concentrating on knowledge (its representation) and reasoning (logic inference), autonomous agents focus on fast, reactive behavior, adaptation and learning. An autonomous agent is situated within some environment, perhaps our real world, or an artif icial environment withing a computing system: an operating system, a database, or a network. The agent actively senses its environment and acts upon its environment in the service of its own agenda. The basic question is, how does an ensemble of agents produce intelligent or interesting output in a coordinated fashion? Autonomous agents can be subdivided into biological, robotic, artif icial life and software agents. Typical representatives of adaptive autonomous agents include Brooks subsumption architecture [3], Maes behavior networks and competence modules [22], or Moravec bush robots [26] Probably, the most serious candidates for formal models to study and design autonomous agents are -calculus [24, 25], interaction machines [31, 32], and (random) automata networks [14]. We will concentrate only on automata networks. Automata networks are a generalization of cellular automata, consisting of a cellular space in the form of an arbitrary digraph and an associated family of f inite-state machines (only f initely many of which are distinct). An arbitrary automata network can be expressed in $-calculus in a way very similar to simulation of cellular automata (each node i will have its own transition rules i and di of neighbors, instead of common and d from cellular automata). Thus we can conclude that $-calculus is rich enough to study dynamic world of autonomous agents.
We can consider intelligent autonomous agents as distributed AI systems built usually of simple, non-intelligent local components, whose activity may result in a complex global emergent behavior. It should be clear that emerging intelligent behavior of autonomous agents is the result of adaptation. Given this fact, it becomes important that the quality of adaptation, learning can and should be expressed and measured formally. This is done using $-calculus cost functions. The beauty and power of the cost approach is that it provides a uniform mechanism for the expression of control (by numerical feedback), optimization, learning, adaptation, uncertainty, incompleteness or vagueness (costs interpreted as probabilities, or as a fuzzy-set membership function), classif ication errors in neural networks, f itness from genetic algorithms, the minimum description length, entropy, level of interestingness, energy level, distance, time, time or space complexity, etc. In terms of the $-calculus, adaptive autonomous agents consist of a parallel composition of component agents
( k i Ai ); where (:= (Ai ) Pi ) is a def inition of the i-th agent with body Pi .
Each agent has its own meta-control (a simple modifying algorithm) allowing to satisfy its local goal(s) or drive(s), and its own or common cost function (used for goal achievement, adaptation and learning). The above means that the cost function for n agents consists of a vector of n component cost functions. It may exist also a global cost function (if we know what we want to achieve). Cost functions represent feedback (reward/penalty/sensory signals) from the environment. Agents communicate with other agents and sense environment by message passing ($-calculus send !, mutation 7! and receive operators). To avoid a combinatorial explosion, on-line optimization can involve a restricted subset of agents, and many or only a few steps of execution. The problem is reduced to f inding an appropriate objective cost function(s), and the choice of the optimization parameters, methods, and algorithms. Because only “simple” local optimizations are performed, the complexity of global (perhaps even intractable) optimization is transformed to a set of simple (and feasible) local optimizations. The most challenging and interesting question concerns which behaviors (local optimizations) should be performed by individual agents and how to orchestrate their co-operation (topologies and methods of interactions), to achieve the desired global effect. Technically this is resolved in $-calculus by making unobservable part of the world (representing an environment and other agents) for a given agent in the solution of k optimization problems. The idea behind this optimization is to provide a f lexible approach allowing performance of local or global optimization in time or space. This is done by replacing part of $-expressions (note, in $-calculus everything is a $-expression: agents, their behavior, data, interactions, environment, cost functions) by invisible silent $-expressions, which allow shutting down the part of the world either spa-
tially or temporally. For observation congruence, silent $expressions have neutral cost 0, thus they do not participate directly in optimization. However, indirectly they are an integral part of $-expressions, and thus inf luence the results. For strong congruence, silent actions (representing incomplete local knowledge of an agent) are still invisible, but their cost are estimated and different from zero. Depending on the quality of cost estimation, the global optimization (emerging global behavior, achieving global goal, avoiding chaotic behavior) can be either successful or not. The crucial question of producing intelligent or interesting output, in terms of $-calculus, is transferred to a more measurable problem: how global optimization can emerge from local vector optimization (pending that the global cost function can be unknown)? $-calculus has been applied to the Off ice of Naval Research SAMON robotics testbed to derive GBML (Generic Behavior Message-passing Language) for behavior planning, control and communication of heterogeneous Autonomous Underwater Vehicles (AUVs) [12]. 6.4 DNA-Based Computing The notion of using DNA molecules to do computing originated in a paper by Adleman [1] who solved the NP-complete Hamiltonian path problem in a test tube. Litpon showed how to use the same primitive DNA operations as Adleman to solve directly any SAT problem [20] and Beaver proved that the class of problems solved in a feasible way using Adleman’s methods is PSPACE [2]. Both Beaver and independently Smith [2, 29] showed how to construct a molecular nondeterministic Turing machine, i.e. that molecular computers are Turing computation universal. DNA-based computing in the Adleman’s style implements the class of massively parallel randomized (probabilistic) algorithms. It consists of three basic operations: encoding that maps the problem onto alphabet of nucleic acid bases fA; T; G; C g, hybridization and ligation which perform the basic core processing, and extraction that makes the results available for humans. Hybridization is a chemical process that joins two complementary single strands into a double strand such that A binds with T and C with G. In $-calculus hybridization of the string TAGC with string ATCG can be described as ( k ( (! (a T )) (! (t A)) (!
(c G)) (! (g C ))) ( ( (a XT )) ( (t XA )) ( (c XG)) ( (g XC )) (A XT ) (T XA ) (C XG) (G XC )))
which leads after 4 communications through send and receive channels a,t,c,g to a double string ( k ( (A T ) (T A) (C G) (G C ))). We can also simulate incomplete hybridization and hybridization with errors. Ligation is a chemical process where two double strands are joined into one double strand. In $-calculus ligation of a double string dstring1 with double string dstring2 can be described as ( k (! (a dtsring2 )) ( (
(a X )) dstring1 X ) resulting in ( dtsring1 dstring2 ) after communication of both strings through channel a. Cost functions can be used in two roles: for quality of encoding by def ining metrics like in [15] to minimize errors of hybridization, and as a modifying algorithm driving force to direct a set of biomolecular reactions in the direction of minimal Gibbs free energy or entropy [30, 7].
7 Conclusions We believe that the process algebra approach could be useful for the future development of the several evolutionary computation subareas. Encoding in $-calculus should be considered as preliminary steps for studying emerging behavior for multiple agents, and hopefully giving some light on problems of construction universality, self-reproduction and evolution. We hope that this approach will lead to new results on convergence and necessary and suff icient conditions for global optimizations by local interactions. Standardization and separability of cost metrics should be benef icial for evolutionary computation to control the length of generated solutions.
Bibliography [1] Adleman L.M., Molecular Computation of Solutions to Combinatorial Problems, Science, vol.266, 1994, 1021-1024. [2] Beaver D., A Universal Molecular Computer, DNA Based Computers, DIMACS Series in Discrete Mathematics and Theoretical Computer Science, vol.27, 1996, 29-36. [3] Brooks R.A., Elephants Don’t Play Chess, in P.Maes (ed.), Designing Autonomous Agents: Theory and Practice from Biology to Engineering and Back, The MIT Press, 1994, 3-15. [4] Burks A., Essays on Cellular Automata, Univ. of Illinois Press, 1970. [5] Church A., The Calculi of Lambda Conversion, Princeton, N.J., Princeton University Press, 1941. [6] Dean T., Boddy M., An Analysis of Time-Dependent Planning, in Proc. Seventh National Conf. on Artif icial Intelligence, Minneapolis, Minnesota, 1988, 49-54. [7] Deaton R. et al, Information Transfer through Hybridization Reactions in DNA based Computing, in Proc. of the Second Annual Genetic Programming Conference GP-97, Morgan Kaufmann, Stanford University, 1997, 463-471. [8] Eberbach E., SEMAL: A Cost Language Based on the Calculus of Self-modif iable Algorithms, Intern. Journal of Software Engineering and Knowledge Engineering, vol.4, no.3, 1994, 391-408. [9] Eberbach E., A Generic Tool for Distributed AI with Matching as Message Passing, Proc. of the Ninth IEEE Intern. Conf. on Tools with Artif icial Intelligence TAI’97, Newport Beach, California, 1997, 11-18. [10] Eberbach E., Enhancing Genetic Programming by $-calculus, Proc. of the Second Annual Genetic Programming Conference GP-97, Morgan Kaufmann, Stanford University, 1997, 88. [11] Eberbach E., Brooks R., $-Calculus: Flexible Optimization and Adaptation under Bounded Resources in Real-Time Complex Systems, Applied Research Laboratory, The Pennsylvania State University, 1999.
[12] Eberbach E., Brooks R., Phoha. S., Flexible Optimization and Evolution of Underwater Autonomous Agents, in: New Directions in Rough Sets, Data Mining, and Granular-Soft Computing, LNAI 1711, Springer-Verlag, 1999, 519-527. [13] Eberbach E., Expressiveness of $-Calculus: What Matters?, in: Advances in Soft Computing, Proc. of the 9th Intern. Symp. on Intelligent Information Systems IIS’2000, Bystra, Poland, Springer-Verlag, June 12-16, 2000. [14] Garzon M.,Models of Massive Parallelism: Analysis of Cellular Automata and Neural Networks, Springer-Verlag, 1995. [15] Garzon M. et al, A New Metric for DNA Computing, in Proc. of the Second Annual Genetic Programming Conference GP-97, Morgan Kaufmann, Stanford University, 1997, 472-478. [16] Holland J., Adaptation in Natural and Artif icial Systems, second edition, The MIT Press, 1992. [17] Horvitz E., Reasoning about Beliefs and Actions under Computational Resources Constraints, in Proc. of the 1987 Workshop on Uncertainty in Artificial Intelligence, Seattle, Washington, 1987. [18] Koza J., Genetic Programming I, II, III, The MIT Press, 1992, 1994, 1999. [19] Langton Ch., Artificial Life: An Overview, The MIT Press, 1996. [20] Lipton R., DNA Solutions of Hard Computational Problems, Science, vol.268, 1995, 542-545. [21] Lohn J., Self-Replication Systems in Cellular Space Models Tutorial, Genetic Programming Conference GP97, Stanford University, 1997. [22] Maes P. (ed.), Designing Autonomous Agents: Theory and Practice from Biology to Engineering and Back, The MIT Press, 1994. [23] Michalewicz Z., Genetic Algorithms + Data Structures = Evolution Programs, third edition, Springer-Verlag, 1996. [24] Milner R., Parrow J., Walker D., A Calculus of Mobile Processes, I & II, Information and Computation 100, 1992, 1-77. [25] Milner R., The Polyadic -Calculus: A Tutorial, in F.L.Bauer,W.Brauer (eds.) Logic and Algebra of Specif ication, Springer-Verlag, 1992, 203-246. [26] Moravec H., Mind Children, Harvard Univ. Press, 1988. [27] Russell S., Norvig P. (1995) Artificial Intelligence: A Modern Approach, Prentice-Hall. [28] Sipper M., Evolution of Parallel Cellular Machines: The Cellular Programming Approach, Springer-Verlag, 1997. [29] Smith W.D., DNA Computers in Vitro and Vivo, DNA Based Computers, DIMACS Series in Discrete Mathematics and Theoretical Computer Science, vol.27, 1996, 121-185. [30] Stryer L., Biochemistry, W.H.Freeman and Company, 4th ed., 1995. [31] Wegner P., Why Interaction is More Powerful Than Algorithms, CACM, May 1997, vol.40, no.5, 81-91. [32] Wegner P., Interactive Foundations of Computing, Theoretical Computer Science, 1997, (also http://www.cs.brown.edu/people/pw). [33] Wegner P., Goldin D., Coinductive Models of Finite Computing Agents, Electronic Notes in Theoretical Computer Science, vol.19, 1999 (also http://www.cs.umb.edu/˜dqg).
