A Distributed Neural Network Architecture for ... - Semantic Scholar

Communicated by Rodney Brooks

A Distributed Neural Network Architecture for Hexapod Robot Locomotion Randall D. Beer Departments of Computer Engineering and Science and Biology, Case Western Reserve University, Cleveland, OH 44106 U S A

Hillel J. Chiel Departments of Biology and Neuroscience, Case Western Reserve University, Cleveland, OH 44106 U S A

Roger D. Quinn Kenneth S. Espenschied Department of Mechanical and Aerospace Engineering, Case Western Reserve University, Cleveland, OH 44106 U S A

Patrik Larsson Department of Electrical Engineering and Applied Physics, Case Western Reserve University, Cleveland, OH 44106 U S A

We present a fully distributed neural network architecture for controlling the locomotion of a hexapod robot. The design of this network is directly based on work on the neuroethology of insect locomotion. Previously, we demonstrated in simulation that this controller could generate a continuous range of statically stable insect-like gaits as the activity of a single command neuron was varied and that it was robust to a variety of lesions. We now report that the controller can be utilized to direct the locomotion of an actual six-legged robot, and that it exhibits a range of gaits and degree of robustness in the real world that is quite similar to that observed in simulation. 1 Introduction

Even simpler animals are capable of feats of sensorimotor control that exceed those of our most sophisticated robots. Insects, for example, can walk rapidly over rough terrain with a variety of gaits and can immediately adapt to changes in load and leg damage, as well as developmental changes (Graham 1985). Even on flat horizontal surfaces, insects walk with a variety of different gaits at different speeds (Wilson 1966). These gaits range from the wave gait, in which only one leg steps at a time in a back-to-front sequence on each side of the body (this sequence is called Neural Computation 4,356-365 (1992) @ 1992 Massachusetts Institute of Technology

Neural Network Architecture for Hexapod Robot Locomotion

357

Figure 1: A comparison of simulated and robot gaits. Black bars represent the swing phase of a leg and the space between bars represents its stance phase. (Top) Leg labeling conventions. (Left) Selected gaits observed in simulation as the activity of the command neuron is varied from lowest (top) to highest (bottom) (Beer 1990). (Right) Gaits generated by the robot under corresponding conditions. Here the duration of a swing bar is 0.5 seconds. a metachronal wave), to the tripod gait, in which the front and back legs on each side of the body step in unison with the middle leg on the opposite side (see left side of Fig. 1). While most current research in legged robot locomotion utilizes centralized control approaches that are computationally expensive and brittle, insect nervous systems are distributed and robust. What can we learn from biology? In previous work (Beer et at. 1989), w e described a neural network architecture for hexapod locomotion. The design of this network was based on work on the neuroethology of insect locomotion, especially Pearson's flexor burst-generator model for walking in the American cockroach (Periplaneta americana) (Pearson et al. 1973; Pearson 1976). Through simulation, we demonstrated that this network was capable of generating a continuous range of statically stable gaits similar to those observed

358

Randall D. Beer et al.

in insects (see left side of Fig. 11, as well as smooth transitions between these gaits. The different gaits were produced simply by varying the tonic level of activity of a single command neuron. In addition, a lesion study of this network demonstrated both its surprising robustness and the subtlety of the interaction between its central and peripheral components (Chiel and Beer 1989). A natural question to ask is whether these results were just artifacts of the many physical simplifications of the simulation or whether they are robust properties of the network that persist in the presence of such physical realities as delay, friction, inertia, and noise. This is a difficult question to resolve given the subtle dependencies of this controller on sensory feedback (Chiel and Beer 1989). The only way to determine whether this distributed controller had any practical utility was to design and build a six-legged robot and interface it to the locomotion network. 2 Locomotion Controller

The circuit responsible for controlling each leg is shown in Figure 2. Each leg controller operates in the following manner: Normally, the foot motor neuron is active (i.e., the leg is down and supporting weight) and excitation from the command neuron causes the backward swing motor neuron to move the leg back, resulting in a stance phase. Periodically, this stance phase is interrupted by a burst from the pacemaker, which inhibits the backward swing and foot motor neurons and excites the forward swing motor neuron, resulting in a swing phase. The time between bursts in the pacemaker, as well as the velocity output of the backward swing motor neuron during a stance phase, depend on the level of excitation provided by the command neuron. In addition, sensory feedback is capable of resetting the pacemaker neuron, with the forward angle sensor encouraging the pacemaker to terminate a burst when the leg is at an extreme forward position and the backward angle sensor encouraging the pacemaker to begin a burst when the leg is at an extreme backward position. There are six copies of the leg controIler circuit, one for each leg, except that the single command neuron makes the same two connections on each of them. Following Pearson’s model, the pacemakers of all adjacent leg controllers mutually inhibit one another, discouraging adjacent legs from swinging at the same time (Fig. 3). At high speeds of walking, this architecture is sufficient to reliably generate a tripod gait. However, at lower speeds of walking, the network is underconstrained, and there is no guarantee that the resulting gaits will be statically stable. To enforce the generation of metachronal waves, we added the additional constraint that the natural periods of the pacemakers are arranged in a gradient, with longer periods in the back than in the front (Graham 1977). Under these conditions, the pacemakers phase-lock into a stable metachronal


359

Command Backward Angle Sensor

r

Swing

-t

Pacemaker

uw Excitatory Connection

ForwardAngle Sensor

c Inhibitory Connection

Figure 2: The leg control circuit. Each leg is monitored by two sensory neurons that signal when it has reached an extreme forward or backward position. Each leg is controlled by three motor neurons responsible for the state of the foot, the velocity with which the leg swings forward, and the velocity with which the leg swings backward, respectively. The motor neurons are driven by a pacemaker neuron whose output rhythmically oscillates. A single command neuron makes the same two connections on every leg controller. The architecture also includes direct connections from the forward angle sensor to the motor neurons, duplicating a leg reflex known to exist in the cockroach. The state of each neuron c j ~ j i f j ( V j ) INTi EXTi, is governed by the equation CidVildt = -Vi/Ri where Vi, Ri, and Ci, respectively, represent the voltage, membrane resistance, and membrane capacitance of the ith neuron, wji is the strength of the connection from the jth to the ith neuron, f is a saturating linear threshold activation function, and EXTi is the external current injected into the neuron. INTi is an intrinsic current present only in the pacemaker neurons that causes them to oscillate. This current switches between a high state of fixed duration and a low state whose duration depends linearly on the tonic level of synaptic input, with excitation decreasing this duration and inhibition increasing it. In addition, a brief inhibitory pulse occurring during a burst or a brief excitatory pulse occurring between bursts can reset the bursting rhythm of the pacemaker.

+

+

+

360


Figure 3: The pacemaker neurons of adjacent leg controllers are coupled by mutual inhibition. relationship. We chose to enforce this constraint by making the range of motion of the rear legs slightly larger than that of the middle legs, whose range of motion in turn is slightly larger than that of the front legs. A complete discussion of the design of this network and its relationship to Pearson’s model can be found in Beer (1990). 3 Robot

To examine the practical utility of this locomotion controller, we designed and built a six-legged robot (Fig. 4, top). The network was simulated on a personal computer using the C programming language and interfaced with the robot via A / D and D/A boards. Because the controller was originally designed for a simpler simulated body (see top of Fig. l),two main issues had to be addressed in order to connect this controller to the robot. First, the locomotion controller assumes that the swing and lift motions of the leg are independent, whereas in the robot these two degrees of freedom are coupled (Fig. 4, bottom). In simulation, this problem was dealt with by having a stancing leg passively stretch between its joint and foot. For the robot to maintain a constant height (h) above the ground, the radial length (Y) of a stancing leg must be adjusted by the simple kinematic transformation Y = h/ cos 0.


361

Figure 4: (Top) The hexapod robot. Its dimensions are 50 cm long by 30 cm wide and it weighs approximately 1 kg. (Bottom) Each leg has two degrees of freedom: an angular motion responsible for swing and stance movements and a radial motion involved in raising and lowering the leg. The swing motion has a range of over 45" from vertical in either direction. The radial motion is accomplished by means of a rack-and-pinion transmission. Both degrees of freedom are driven by 2 W DC motors with associated integral transmissions. Position sensing for each degree of freedom is provided by potentiometers mounted in parallel with the motors.

362


A second compatibility issue derives from the simplified physics utilized in the original simulation, in which the activity of the forward and backward swing motor neurons was translated directly into velocities (Beer 1990). To interface the output of the neural network controller to the physical dynamics of the robot, we made use of the equilibrium point hypothesis for motor control (for review see Bizzi and Mussa-Ivaldi 1990). This hypothesis states that the nervous system generates limb trajectories not by directly specifying joint torques but rather by specifying a sequence of equilibrium positions known as a virtual trajectory. This hypothesis is based on the following two facts: (1) muscles have springlike properties that are in equilibrium when the torques generated by opposing muscles exactly cancel and (2) neural input to a muscle has the effect of selecting a length/ tension curve and therefore an equilibrium position for the limb as a whole. The equilibrium point hypothesis suggests the following approach. The velocity output of the network is integrated to obtain a sequence of virtual positions. These virtual positions are then translated (via the trigonometric transformations described above) into sequences of positions of the swing and lift motors. Finally, these swing and lift positions are fed into position controllers that drive the corresponding motors with voltages proportional to the deviations between the actual positions and the desired positions. These position controllers are implemented in analog circuitry for speed. Because the motors are backdrivable, this scheme also gives a spring-like property to the legs that lends stability to the robot.

4 Results and Discussion

Under the control of the locomotion network, the robot exhibits a range of gaits similar to those observed in simulation as the command neuron activity is changed (see right side of Fig. 1). These gaits range from ones in which distinct metachronal waves are readily distinguished at low speeds of walking to the tripod gait at high speeds of walking. Within this range of gaits, the robot's speed of progression varies from 4.5 to 8.3 cm/sec. In addition, we studied the effects on the robot's walking of a number of lesions that were previously performed in simulation (Chiel and Beer 1989). In all cases, the response of the physical robot was quite similar to what we had previously observed in simulation (Chiel et al., in press). The controller was able to cope with the removal of such components as single sensors, a small fraction of the coupling connections between pacemakers, or the command neuron to pacemaker connections. In addition, we found that the robot was capable of reflex stepping. If the command neuron is disabled but the robot is steadily pushed forward at different speeds by externally updating the position


363

controllers, then it still exhibits the full range of gaits. Thus it appears that the continuous range of gaits is a robust property of the locomotion network and not simply an accident of simulation. Interestingly, this robotic implementation did reveal one weakness of the locomotion controller that we did not think to examine in simulation. While we have found the controller to be quite robust in general to the delays inherent in the physical robot, it is sensitive to asymmetic delays that cause the legs on one side of the body to consistently lag behind those on the opposite side. These asymmetric delays are due to the inevitable variations in the response characteristics of the electrical and mechanical components of the robot. In the presence of such asymmetric delays, the crossbody phasing of the legs is disturbed. Once we identified this problem, however, a simple adjustment to the stiffnesses of the position controllers, which affects the amount that a leg lags behind its prescribed position, restored the proper phasing. Nevertheless, discoveries such as this justify the effort involved in undertaking a robotic implementation. Brooks has described a partially distributed locomotion controller for a six-legged robot known as Genghis (Brooks 1989). This robot is controlled by a network of finite state machines augmented with registers and timers. In Brooks’ locomotion controller, the basic swing/stance cycle of each leg is driven by a chain of reflexes involving only coarse local information about leg position and load. For example, whenever a leg is lifted for some reason, it reflexively swings forward and whenever one leg swings forward, all other legs move backward slightly. With elaborations of this basic controller, the robot not only sqcessfully walked, but could also negotiate small obstacles and follow slowly moving objects with infrared sensors. However, Brooks’ controller is not as fully distributed as the network architecture described in this paper. In Genghis, the movements of the individual leg controllers are coordinated by a single, centralized finite state machine which tells each leg when to lift. Different gaits are generated by moddying this machine. While Maes and Brooks (1990) have recently described a version of this controller that does not require a centralized gait sequencer, their new controller is capable of generating only the tripod gait. In contrast, our neural network controller generates a continuous range of gaits without any centralized gait sequencer. Instead, the different gaits result from the dynamics of interaction between the pacemaker neurons controlling each leg and the sensory feedback that they receive. The architecture described in this paper focuses solely on the problem of sequencing leg movements, so as to maintain static stability at a variety of walking speeds during straight-line locomotion across flat, horizontal surfaces. Of course, gait control is only one aspect of legged locomotion. Other important issues include postural control, turning, negotiation of complex terrain, and compensation for leg damage or loss. In future work, we plan to draw on further studies of insect locomotion and its

364


neural basis to address these additional issues (Burrows 1989; Pearson and Franklin 1984; Cruse 1990). If we had taken a classical control approach to legged locomotion, it is unlikely that a distributed architecture such as the one we have presented here would have resulted. We believe that our results represent a simple example of a very powerful idea: neural network architectures abstracted from biological systems can be directly applied to the control of autonomous agents (Beer 1990). Because they have evolved over a significant period of time, biological control systems are much more flexible and robust than their engineered counterparts. However, they are also much more difficult to understand. Simulation can serve as a n important intermediate step in the process of abstracting a biological control principle. On the other hand, only a physical implementation in an actual robot can prove such a principle’s practical utility. In this paper, we have demonstrated that our distributed locomotion controller is a viable approach to hexapod robot wallung.

Acknowledgments This work was supported by Grant N00014-90-J-1545 to R. D. B. from the Office of Naval Research and Grant NGT-50588 from NASA Goddard. Additional support was provided by the Cleveland Advanced Manufacturing Program through the Center for Automation and Intelligent Systems Research and the Howard Hughes Medical Institute. H. J. C. gratefully acknowledges the support 07 the NSF through Grant BNS8810757.

References Beer, R. D. 1990. Intelligence as Adaptive Behavior: An Experiment in Computational Neuroethology. Academic Press, San Diego. Beer, R. D., Chiel, H. J., and Sterling, L. S. 1989. Heterogeneousneural networks for adaptive behavior in dynamic environments. In Advances in Neural lnformation Processing Systems 1 , D. S . Touretzky, ed., pp. 577-585. Morgan Kaufmann, San Mateo, CA. Bizzi, E., and Mussa-Ivaldi, F. A. 1990. Muscle properties and the control of arm movement. In An lnvitation to Cognitive Science, Volume 2: Visual Cognition and Action, D. N. Osherson, S. M. Kosslyn, and J. M. Hollerbach, eds., pp. 213242. MIT Press, Cambridge, MA. Brooks, R. A. 1989. A robot that walks; emergent behaviors from a carefully evolved network. Neural Comp. 1(2), 253-262. Burrows, M. 1989. Processing of mechanosensory signals in local reflex pathways of the locust. ]. Exp. Biol. 146, 209-227.


365

Chiel, H. J., and Beer, R. D. 1989. A lesion study of a heterogeneous neural network for hexapod locomotion. Proc. Int. J. Conf. Neural Networks [IJCNN 891, I, 407-414. Chiel, H. J., Beer, R. D., Quinn, R. D., and Espenschied, K. S. In press. Robustness of a distributed neural network controller for locomotion in a hexapod robot. To appear in IEEE Transactions on Robofics and Automation. Cruse, H. 1990. What mechanisms coordinate leg movement in walking arthropods? Trends Neurosci. 13(1), 15-21. Graham, D. 1985. Pattern and control of walking in insects. Adu. Insect Physiol. 18, 31-140. Graham, D. 1977. Simulation of a model for the coordination of leg movements in free walking insects. Biol. Cybernet. 26, 187-198. Maes, P., and Brooks, R. A. 1990. Learning to coordinate behaviors. Proc. Eighth Natl. Conf. AI [AAAI-90], 796-802. Pearson, K. G. 1976. The control of walking. Sci. Am. 235, 72-86. Pearson, K. G., Fourtner, C. R., and Wong, R. K. 1973. Nervous control of walking in the cockroach. In Control of Posture and Locomotion, R. 8. Stein, K. G. Pearson, R. S. Smith, and J. B. Bedford, eds., pp. 495-514. Plenum Press, New York. Pearson, K. G., and Franklin, R. 1984. Characteristics of leg movements and patterns of coordination in locusts walking on rough terrain. Int. J. Robotics Res. 3(2), 101-112 Wilson, D. M. 1966. Insect walking. Annu. Rev. Entomol. 11, 103-122.

Received 9 July 1991; accepted 9 October 1991.