View Simulation System : A Mobile Robot Simulator

0 downloads 0 Views 171KB Size Report
including vision processing for our real mobile robots. To show the ..... the walls, floor and ceiling results in incorrect textures. ... to an exactly straight trajectory under human opera- tion. .... Soccer Server Manual, rev.1.03 edition,. 1996.
View Simulation System : A Mobile Robot Simulator using VR Technology Yoshio Matsumoto† , Takeshi Miyazaki‡, Masayuki Inaba‡ and Hirochika Inoue‡ † Graduate School of Information Science, Nara Institute of Science and Technology 8916-5, Takayamacho, Ikoma-city, Nara, 630-0101, Japan ‡ Department of Mechano-Informatics, Faculty of Engineering, The University of Tokyo Email: [email protected] Abstract Simulation is an important issue in robotics research because it is essential for evaluating and predicting the behavior of a robot. However conventional simulators don’t have the function of simulating the visual perception process. This indicates that it is impossible to simulate the vision-based behavior of a robot. In this paper, we propose a new simulation system “View Simulation System (VSS),” which is designed to simulate the behavior of a mobile robot equipped with vision. VSS has CG generator and vision processing hardware, and can run any programs including vision processing for our real mobile robots. To show the feasibility of the simulator, we applied it to the evaluation of our visual navigation method.

1

Introduction

In order to study an algorithm for generating a behavior for a robot, it is important to make a quantitative evaluation of the algorithm through many trials of experiments while changing parameters. Making such trials is a hard task in the real environment, and simulators are invented to perform a “virtual experiment”. However conventional simulators for mobile robots have some limitations. They usually assume much simpler environment than reality and the action of the robot is generated granting that the necessary sensory information is all known. Such systems don’t take the sensor-based perception stage into account. An algorithm to generate the behavior of mobile robots in the real world depends on the sensors’ type, accuracy and so on, which means algorithms for perceiving sensory information and generating behaviors cannot be separately evaluated. For instance, vision based mobile robots should be simulated with the vision processing, however this is impossible to be performed on conventional simulators. In this paper, we propose a new simulation system “View Simulation System (VSS),” which is designed to simulate a behavior of a mobile robot equipped with

vision. VSS has CG generator and vision processing hardware, and can run the same program as one for our real mobile robot, including vision processing. To show the feasibility of VSS, we applied it to the evaluation of our visual navigation method.

2

Simulation Using Virtual Environment

A robot perceives the outer world, then generates a behavior and physically acts in the world. A simulator is regarded as a system that virtually performs these programs within a computer. However, most of the conventional simulators omit the perception process based on the virtual sensory information, and obtains the recognized data directly from the internal state of the virtual world. The simulator league of the RoboCup[1], which is recently attracting attentions as a test bed for artificial intelligence, has a scheme to make the simulation more realistic. It takes some uncertainty or imperfectness in the perception stage into account. For instance, each robot has a certain limited field of view, and the obtained data have noises at a certain rate. However since the perception process (image processing) can not be used in the simulator, the program for a robot in middle size league cannot be simulated in the simulator league. In this paper, we propose a simulation system that can simulate a robot’s behaviors including the image processing. Figure 1 indicates the concept of the system. Figure 1 (1) is a processing loop in the real robots, while Figure 1 (2) is the processing loop in a conventional simulator which omits the recognition process. Figure 1 (3) is the outline of the View Simulation System, which enables the simulation of both the perception process and the behavior generation process. It has been quite common in the computer vision research to utilize a computer generated image (CG) as a target for the image processing to test a new algorithm. For example, an algorithm for generating depth

(1) Real Robot

Robot sensor data

Environment

Onyx RE2

Sensor Perception Silic

Comput onG raph er Systems ics

interaction

recognized data action

Robot Body

Vision Processing System

ON YX

Action Generation

NTSC

(2) Traditional Simulator Virual World

InmosLink Ethernet

recognized data

Environment interaction

action

Robot Body

Action Generation

SparcStation5

Fig. 2 : Overview of View Simulation System.

(3) Proposed Simulator Virual World Environment

sensor data

interaction

Robot Body

Sensor Perception recognized data

action

Action Generation

Fig. 1 : Outline of the processing in real robot system (1), conventional simulator (2) and VSS (3).

and optic flow is evaluated using CG generated from known 2D or 3D models [2]. Demetri [3] applied an interactive 3D CG for vision processing to generate an algorithm for a virtual fish’s behavior. In this research, the virtual view from the fish’s eye is used in image processing, and the action of the virtual fish is fed back to update the virtual world. This is an interesting use of the CG in that the system utilizes the CG not for visualization but for image processing and visual feedback. However the system is closed in the virtual world, and neither the real time operation nor the application to the real robot is taken into account. In robotics research, CG has been utilized only for visualization, and no system has utilized it to realize a feedback loop including the vision processing in simulation. This is because it has been regarded that the generation of 3D CG in high quality requires enormous computational cost, and the modeling of the real environment is very difficult. However owing to the rapid progress in computer hardware and software for VR applications, it has become possible to generate CG in high quality in real time. In this paper, we describe a simulation system including the vision process which utilizes a graphics workstation for the virtual reality applications, and apply it to evaluate our visual navigation method.

3

View Simulation System

Figure 2 and Figure 3 illustrate the hardware and software configuration of the system. It consists of (1) the CG Generator, (2) the Image Processor and (3) the Interface, and is designed to enable for application programs for a real robot to run without any modification. The CG Generator contains the models of the robot and the environment. It simulates the movement of the robot and produces the virtual view from the robot’s eye. The Image Processor is equivalent to the vision processing system which is used for the real robot system, and the application program runs on this processor. The Interface connects the Image Processor (the application program) with the CG Generator (the simulation system). A CG of 3D virtual environment generated by the CG Generator based on the model of the environment and the robot is fed into the Image Processor and is used in the perception process. The Image Processor then outputs an action command, which is sent to the CG Generator via the Interface. The internal state (position of the robot etc) in the CG Generator is updated based on this command. This cycle forms the feedback loop for real time simulation. 3.1

CG Generator

The generation of CG requires a high computational cost in general. Due to the requirement of real time CG generation, we adopted a graphics workstation OnyxRealityEngine2 (SGI). It has a video output as well as a RGB output, which is utilized to feed the generated CG to the Image Processor. The CG Generatior contains 3D models of the environment and the robot to produce a virtual robot’s view image, and updates the models after receiving an action command from the Image Processor via the Interface. Other sensory information such as shaft en-

Vision Processing System

ONYX RE2 Virtual 3D World Sensor Emulator

Renderer

view

Mobile base Emulator

Vision Processing & Control Program for Real Robot Robot Interface Emulator

action sensory data

iserver

action

iserver

sensory data

User Interface

SparcStation 5

Fig. 3 : Software configuration of VSS.

coder of the wheels can be also sent to the Image Processor when requested. OpenInventor is used as the API to generate the CG’s. It is built on top of OpenGL, providing programming a model and user interface for OpenGL. Since the data structure of the model of OpenInventor is equivalent to that of VRML, commercial modeling tools for VRML can be utilized for building models.

3.2

Image Processor

JSK-IFMMT-01[4] is utilized as the Image Processor. This vision processing system has a Transputer T805, frame memories and a correlation LSI. This is exactly the same system as the one used in the real mobile robot system, and the application program for perception and behavior generation runs on this processor. VSS has source-level compatibility with the real robot system, which means programs developed for the robot can be utilized for simulation without any modification in the source code. In the real robot system, action commands produced by the application program are received by the controller process and are executed immediately. In VSS, they are received by controller emulator process and sent to the CG Generator via the Interface.

3.3

Interface

The Interface part of the system consists of a SPARCStation5 containing a Transputer link board. It is regarded as a host computer for the Transputer JSKIFMMT-01, which is necessary because the Transputer on the Image Processor cannot directly communicate with the CG Generator to give action commands to the virtual robot. It also emulates the user interface such as a keyboard and a mouse as input devices to the robot.

4

Modeling of Virtual Environment

4.1

Environment Model

To simulate a robot’s behavior, the virtual environment should have the similar complexity to the real environment where the real robot behaves. Building a precise geometric model of the existing environment[5] could be an approach for representing a virtual environment. However we adopted a simple geometric model together with the texture on it, because the most important function of the model in VSS is not to represent the complex shape but to represent the complex appearance of the environment. This model can represent a moderate complexity of the environment, while keeping the modeling process easy. However due to the limitation of the hardware and the simplicity of the modeling, only ambient lighting sources are considered, and neither spot lighting, reflection nor shadow are considered. 4.2

Robot Model

Currently two types of mobile robot models are provided. They correspond to our robots with • 1 driving wheel + 1 steering [6] • 2 driving wheel type [7] respectively. To simplify the kinematics of the motion, following conditions are assumed : • No slippage between the wheels and the floor, • No bumps on the floor, • No time delay and error between the motor command and the action, • No inertia with regard to the body and the wheels of the robot. 4.3 4.3.1

Modeling of Corridor Geometric Model

As an example of the modeling of the environment, a corridor model is built. The model consists a simple metric model with five planes and each plane has a texture taken from the real environment. Figure 4 shows the model of a corridor. The width and the height are the same dimensions as the real one. However the depth is shorter than the real length of the corridor. The front plane substitutes the rest of the corridor, which reduces the data size of the texture and enables fast generation of the CG image. The approach to use textures instead of providing an exact geometric 3D model is a common way to produce a realistic CG image in the field of the virtual reality. 4.3.2

Texture Acquisition

To obtain the texture of the corridor, the “panoramic view [8]” like scanning method was used. The

ceiling height

left_wall right_wall

front

4 floor

width

3

depth

2

Fig. 4 : Geometric model of corridor environment.

1

four slits

front

1

2

3

4

right wall

ceiling

Fig. 6 : Reconstructed virtual corridor and sample views from the robot’s eye.

Fig. 5 : Acquisition of the textures in the corridor.

panoramic view is an image generated by collecting vertical linear images on a certain slit in the image plane, and while the camera moves horizontally, a continuous image is acquired. In our case, in order to obtain the texture of the walls, the ceiling and the floor, four slits shown as the top image in Fig.5 are utilized. As the robot moves forward in the corridor, it scans four textures simultaneously. However the panoramic view method requires an accurate linear motion of the camera. Since the camera is mounted on a mobile robot and the motion of the camera is not sufficiently accurate in our system, it is essential to compensate the camera shake. In our method, translation of the input image based on optic flow is utilized to for the compensation, which is similar way of image stabilization method in some commercial cam-coders. Obtained textures are also shown in Fig.5. Each texture has an uniform resolution. Since the four textures are obtained simultaneously, they don’t have any gaps at the joint. However due to the orientation of the camera which is facing forward, the unevenness of the walls, floor and ceiling results in incorrect textures. Significant distortion due to the distortion of the original camera image can also be seen in the obtained texture shown in Fig.5. Figure 6 illustrates an bird’s eye view of the built corridor model and virtual “robot’s eye views” at four

positions inside the model. There are some inconsistent appearance such as reflection on the floor in the bird’s eye view, however the views from the robot looks sufficiently real to be used for image processing.

5

Experiment : Application to View-Based Navigation

In this section, the feasibility of VSS is confirmed through the application to the navigation method based on the memorization of the “View Sequence[7].” Figure 7 illustrates the outline of the navigation method. The robot memorizes a sequence of views in the teaching run under human’s operation. Then in the autonomous run, it recognizes the location by comparing the memorized views and the current view, and performs a navigation along the given route. Both teaching run and autonomous run can be performed either in VSS or in the real environment, therefore there are four combinations of runs as described in Table 1. In the teaching run using the real robot, it is impossible to produce a view sequence which corresponds to an exactly straight trajectory under human operation. However, using VSS enables the production of not only such an ideal view sequence, but also a view sequence with known errors as regarding the position and the orientation of the robot.

1. Recording Run

real run run in VSS

Memorize views along the route

displacement

[cm] 20 10 0 0

500

1000

1500 [cm]

distance

View-Sequenced Route Representation

left

right

end

2. Autonomous Run matching

action

Fig. 8 : Evaluation of the accuracy of VSS using real View Sequence.

Current view

duce more accurate trajectories by reducing the simplification. However the complexity of the modeling and the accuracy of the trajectories is a trade-off, and we regard the accuracy obtained in this experiment as sufficient for simulation. 5.2

Fig. 7 : Outline of visual navigation method using View Sequence.

(a) (b) (c) (d)

Teaching Run Real Env Real Env VSS VSS

Auto Run Real Env VSS Real Env VSS

Table 1 : Four combinations of runs using VSS.

5.1

Accuracy Evaluation of VSS

In this experiment, the accuracy of VSS in terms of the trajectory in the navigation was tested. In the real teaching run, the robot obtained a view sequence while being given unexpected errors. Then the real autonomous run and the virtual autonomous run are both performed using the same View Sequence. These two runs correspond to (a) and (b) in Table 1. The trajectories in the real environment and the virtual environment are shown as a solid line and a dotted line respectively in Fig.8. Since the teaching run was not exactly straight, the trajectory in the real environment gradually gets closer to the wall with some ruggedness. The trajectory in VSS is much smoother, however it also gives the similar tendency of going gradually aside. This experiment indicates that VSS cannot reproduce the small ruggedness of the trajectory which appears in the real environment, although it can produce the similar tendency of the trajectory as a whole. The difference between those trajectories is caused by the simplification concerning the modeling which was introduced in section 4.2. It may be possible to repro-

Investigation of Errors in Trajectory

In this section, the effect of qualitative errors in terms of the trajectory is investigated using VSS. In the previous experiment, the robot both in the real environment and in VSS indicated a tendency of moving forward while gradually getting closer to the wall. That motion must be caused by some errors in the view memorized in the real teaching run. There could be two sources for the error in the teaching run as follows: • The position of the camera had a lateral translation from the straight trajectory, • The orientation of the camera had a rotational displacement from directing straight forward. By using VSS, qualitative errors can be given in the virtual teaching run, and the trajectory of the autonomous run can be now predicted before performing the autonomous run in the real environment. These experiments correspond to (c) and (d) in Table 1. The view sequence was memorized in VSS while being given qualitative rotational errors as follows: • 3[deg] to the right from 250[cm] to 400[cm], • 3[deg] to the left from 900[cm] to 1100[cm], • no error in other locations. The trajectories of the autonomous run using this view sequence are shown in Fig.9. The dotted line corresponds to the trajectory in VSS and it indicates a trivial errors in the teaching run will result in rather large errors in the trajectory. This result was confirmed in the real environment, and the trajectory is shown as the solid line in Fig.9. The error in the trajectory is at most 6[cm]. 5.3

Improvement of Accuracy in Trajectory

In this experiment, a view sequence along an exactly straight course is obtained in the teaching run in VSS.

displacement

real run run in VSS

[cm] 10

6

0

-10 0

500

1000

1500 [cm]

distance

Fig. 9 : Trajectories using virtual View Sequence with trivial rotational error. using real View Seq. using virtual View seq.

[cm]

displacement

way of modeling according as the recognition method they use.

10 0

-10 0

500

1000

1500 [cm]

distance

Fig. 10 : Improvement of the linearity in autonomous run. Since it doesn’t contain any errors regarding the position and orientation of the camera, it is expected to result in more accurate trajectory than using a normal view sequence. This type of experiment can be regarded as a virtual teaching run using VSS. The dotted and solid lines in Fig.10 illustrate the trajectories in the real autonomous run using a View Sequence obtained in the real environment and in VSS respectively. These trials of navigation correspond to (c) and (a) in Table 1. As is confirmed in the previous experiment, trivial error (e.g. 3 [deg]) in rotation may result in a large displacement from the correct course (e.g. more than 10[cm]). It seems to be impossible for a human to operate a robot with two driving wheels with less errors than 3[deg], which indicates the accuracy of the autonomous run can be improved by using VSS for virtual teaching run. 5.4

Discussion

Since the inertia, the slippage, the bumps and the delay are not taken into account in VSS, resulting trajectories tend to be smoother than real ones. However as a whole, the tendency of the trajectories are similar to each other. This also indicates that the way of modeling fits well for the way of the recognition algorithm. If the robot utilize color information, the textures must have color information in it, and if the robot utilizes metric information, the 3D model of the environment should be more precise. It is impossible to build a virtual environment that fits to any simulation, and the users of the simulator should design the

Conclusion

In this paper, the development and the application of “View Simulation System” which is a simulator for a mobile robot using VR technology was presented. This system consists of a graphics workstation and the same vision processing system that the real robot utilizes. The advantage of VSS is that it can perform the simulation of the behavior of a real robot including vision processing. The way to build an environment for such a system was also presented through building a corridor model as an example. The system was finally applied to evaluate and improve our visual navigation method based on “View Sequence.” The model described in this paper is very simple in terms of the shape. However using more advanced modeling techniques in VR technology will enable wider application for various robots. Recently, the hardware for displaying textured 3D CG is becoming drastically inexpensive in the personal computer market, as 3D computer games are getting more popular. Therefore simulation including visual perception using such hardware will be an important technique in the robotics research.

References [1] Itsuki Noda. Soccer Server Manual, rev.1.03 edition, 1996. [2] D. Marr. Vision. Freeman, 1982. [3] Demetri Terzopoulos, Tamer Rabie, and Radek Grzeszczuk. Perception and Learning in Artificial Animals. In ALife V Proceedings, pages 313–320, 1996. [4] H. Inoue, M. Inaba, T. Mori, and T. Tachikawa. RealTime Robot Vision System based on Correlation Technology. In Proc. of Int’l Symp. on Industrial Robots (ISIR), pages 675–680, 1993. [5] H.Y.Shum, K.Ikeuchi, and R.Reddy. Virtual Reality Modeling From a Sequence of Range Images. In Proc. of IEEE Int’l Conf. on Intelligent Robotics and Systems, pages 703–710, 1994. [6] T. Shibata, Y. Matsumoto, T. Kuwahara, M. Inaba, and H. Inoue. Hyper scooter: a mobile robot sharing visual information with a human. In Proc. of IEEE Conf. on Robotics and Automation, pages 1074–1079, 1995. [7] Y. Matsumoto, M. Inaba, and H. Inoue. Visual Navigation using View-Sequenced Route Representation. In Proc. of IEEE Int. Conf. on Robotics and Automation, pages 83–88, 1996. [8] J.Y.Zheng and S.Tsuji. Panoramic Representation for Route Recognition by Mobile Robot. International Journal of Computer Vision, 9(1):55–76, 1992.