6. AVERT in action, where the blocking car is extracted through the attached bogies. [2] C. Beltrán-González, A. Gasteratos, A. Amanatiadis, D. Chrysostomou,.
The AVERT project: Autonomous Vehicle Emergency Recovery Tool A. Amanatiadis∗ , K. Charalampous∗ , I. Kostavelis∗ , A. Gasteratos∗ , B. Birkicht† , J. Braunstein‡ , V. Meiser§ , C. Henschel§ , S. Baugh¶ , M. Paul¶ and R. May¶ ∗ Democritus
University of Thrace, 12 Vas. Sofias, GR-67100, Xanthi, Greece Im Pfarracker 17/1, D-71723, Grossbottwar, Germany ‡ Force Ware GmbH, Arbachtalstrasse 10, D-72800, Eningen, Germany § Zurich University of Applied Sciences, Technikumstrasse 5, CH-8401, Winterthur, Switzerland ¶ IDUS Consultancy Ltd, 10 Lime Close, RG41 4AW, Wokingham, United Kingdom † BB-Ingenieure,
Abstract—Terrorism threatens horrific loss of life, extensive disruption to city transport and damage to commercial real estate. Vehicles provide an ideal delivery mechanism because they can be meticulously prepared well in advance of deployment and then brought into the area of operations. Furthermore, a real and present danger comes from the threat of Chemical, Radiological, Biological and Nuclear (CRBN) contamination. Current methods of bomb disruption and neutralisation are hindered in the event that the device is shielded, blocked or for whatever reason cannot be accessed for examination. The Autonomous Vehicle Emergency Recovery Tool (AVERT) project introduces a unique capability to Police and Armed Services to rapidly deploy, extract and remove blocking vehicles from vulnerable positions such as enclosed infrastructure spaces, tunnels, low bridges as well as under-building and underground car parks. Within the AVERT project, vehicles can be removed from confined spaces with delicate handling, swiftly and in any direction to a safer disposal point to reduce or eliminate collateral damage to infrastructure and personnel. The overall system will be commanded remotely and shall operate autonomously under its own power and sensor awareness, as a critical tool alongside existing technologies, thereby enhancing bomb disposal response speed and safety.
I.
I NTRODUCTION
Robots for security and intervention operations is a rapidly rising sector of robotics, which employ systems able to support first responders in search and rescue missions [1], [2]. Unfortunately, such robots have not yet been fully adopted by the international emergency response community [3], due to the fact that many technical and scientific challenges still exist in the area of incident response robots. More precisely, they have to be designed for the coarse environmental situations under which they operate in and moreover, they have to be equipped with advanced capabilities and intelligent behaviors to allow them a more independent operation with less need for human supervision [4], [5]. A great amount of research effort is therefore focused on the full adoption of such technology in security, search and rescue missions. The emergency scenarios and operations can be characterized by a vast diversity of operational constraints and conditions, with the initial information gathering task of the emergency scene being a common task in all security and rescue operations [6]. Before the robotic era, the first responders or the Explosive Ordnance Disposal (EOD) teams were responsible for this critical task. The challenges that lie in this particular
mission phase are critical and many times fatal to human personnel. Specifically, both the US National Fire Protection Association and the US National Institute for Occupational Safety and Health have reported that this challenging task is a contributing factor to human casualties [7]. Unfortunately, vehicles can provide an ideal mean for terrorist actions because they can be prepared in advance and then be easily deployed in crowded and busy scenes. Terrorists aim to locate the Vehicle Borne Improvised Explosive Device (VBIED) next to supporting structures for maximum effect (i.e. to cause a building to collapse or to maximize human casualties) and aim to position the vehicle to obstruct the observation, inspection, neutralization and removal of the vehicle by the standard explosive ordnance disposal teams and their robot assets. Routine car park users may inadvertently further hamper efforts to disrupt or neutralize VBIEDs simply by parking third party vehicles to block the access path to the suspect vehicle. The AVERT project is aimed at supporting the explosive ordnance disposal teams dealing with threats from VBIEDs which have been placed by terrorist organizations in proximity to a target building or commercial center. The AVERT project concept is to automate the placing of lifting bogies, capable of omnidirectional movement, under the road wheels of identified vehicles and to synchronize their lifting and path as a group, in order to remove the vehicle without disturbance. II.
AVERT F RAMEWORK
The overall system is comprised of three units and subsystems, namely the Command System, the deployment unit and the bogies. On immediate arrival at the area of operation, the Command Console will be initiated and the deployment unit shall be released with a third party Improvised Explosive Device (IED) robot, controlled from a safe remote location. The AVERT deployment unit (with the lifting bogies stowed onboard the carrier) will be deployed at maximum speed into a position, overlooking the suspect vehicle as well as adjacent vehicles and the car park area. A plan view shall immediately be surveyed and relayed back to the AVERT command console, for user inspection and tasking, whilst the bogie units shall be deployed ready to remove obstructing vehicles, as shown in Fig. 1.
978-1-4799-0880-6/13/$31.00 ©2013 IEEE
Fig. 1. Operational simulation of the AVERT, in which the bogies are released from the deployment unit and autonomously move to under-ride the vehicle (BMW).
The vehicle extraction operation will be a semi-autonomous task. Visual observation and augmented control technologies will enable the human operator to accurately control AVERT. However, AVERT shall navigate its local path and position itself autonomously with speed and accuracy using its sensors to avoid collisions and navigate over or around obstacles. A typical extraction task would be to employ AVERT for the rapid autonomous removal of third party vehicles, which may be blocking and hindering access to the suspect vehicle, moving them swiftly away to a safe position without causing any risk to personnel. The user shall be able to interact with the observation robot to identify and designate the vehicles that are to be moved. Each vehicle should be removed within a short timescale. This operation will be significantly quicker than existing methods not only because of the setup time and sensor autonomy, but also because tasks can be done in parallel (current methods are conducted linearly), thereby reducing total operational time. Furthermore, omnidirectional movement design permits the coordinated AVERT bogie units to manoeuvre the vehicle in any direction. III.
AVERT H ARDWARE A RCHITECTURE
A. Deployment Unit The goal of the deployment unit, apart from carrying the bogies, is to explore the emergency scene and its environment while enriching the human operators with as much as possible scene information. The main challenging factors for such search task are the limited operational time window and the desired optimal response performance. An IED robot will move the deployment unit into a position that overlooks the suspect vehicle and area to which the vehicles shall be extracted. The bogies then will be released by lowering the deployment unit’s frame to the ground and letting the bogies roll off, as shown in Fig 2 . A 3D reconstruction along with video stream from an overview camera will be immediately surveyed and relayed back to the AVERT Command Console [8]. This plan view will be realized by the on-board sensor rig comprised of a SICK laser sensor, a Pan Tilt Unit (PTU) and a digital camera which are fixed to the deployment unit, in a certain height through a mast. The main data processing will be held on the deployment unit which will host also all the batteries and power units.
Fig. 2.
Deployment unit while releasing a bogie.
B. Bogies The bogies shall be controlled autonomously when underriding the vehicles, moving into place from any approach direction to locate the vehicle’s pick up points. In each instance, to extract a vehicle, the bogies will operate as a swarm of autonomous lifting robots, designed to be inserted underneath any vehicle. Since the AVERT bogies have to deal with threats which focus exclusively on urban environment, they will not offer any outdoor capabilities like operating on soft surface, grass or uneven terrain. They will cope with various car park floor types with their typical restrictions and environmental conditions. This well structured operating area allows a high sophisticated drive system design. Omnidirectional movement design, as shown in Fig 3, permits the coordinated AVERT bogie units to manoeuvre the vehicle in any direction (literally the vehicle can be moved sideways or diagonally on the bogies, once the tires have been lifted off the ground). When required to pick up cars parked narrowly to walls, pillars or other cars, the bogie system has to operate within a confined space. The estimated distances between surrounding obstacles and the vehicle to be repositioned are unpredictable, but unobstructed vehicle accessibility is mostly possible from top or from below. Under-riding a vehicle introduces several constraints. Mainly the chassis clearance limits an under-ride type bogie system’s maximum height. A typical ground clearance for private cars is 160mm (within Germany and Austria the minimum ground clearance is not limited but recommended at 110mm). The access to a car from beneath is limited by the track width, when this happens either by front or rear side, and the wheelbase when the access occurs sidewards. Due to the fact that track width is always smaller than wheel base, the vehicle’s track width constitutes the limiting measure for the bogie system’s maximum width. After being positioned underneath the target vehicle, the bogie design offers a commanded split of the bogie in two
(a)
(b)
Fig. 4. a) Registered point clouds after ICP refinement; b) Top-down projection map and the D* Lite path planning.
Fig. 3.
AVERT A-Model Bogie with two Hokuyo laser sensors on-board.
sections, enabling both bogie sections to manoeuvre independently, dock at the vehicle’s opposite wheels of an axle and lift them off the ground by compressing progressively two rollers against the tire, similar to commercially available car movers. The typical gross vehicle weight of passenger cars and light vans is around 2.5 tons and the upper limit by law is 3.5 tons. The bogie systems are considered to provide the capability of lifting and moving vehicles with weights in excess of these figures according to the actual operational deployment. When displacing a lifted vehicle, the bogie’s totally installed driving power is sufficient to move the hooked mass according to the commanded or calculated moving parameters. A prerequisite for omnidirectional movement is that the driving power should split within the bogie System to all bogies, forcing them to have separate main drives for each bogie section. The bogie sections have to operate as a swarm, so power storage is split as well. The main drive system therefore is a battery supply based set of distributed electrical drives. C. Communication Rig AVERT is commanded over a wireless network designed to provide effective connectivity between the elements of the bogie swarm and also to provide planning, situational awareness and status information back to the operator throughout the mission. Separate systems have been specifically developed to provide emergency stop and reversionary control capabilities for the operator throughout the mission. There is provision for incorporating standard low latency coded orthogonal frequency division multiplexing or analogue video supervision with the system to support direct man-in-the loop intervention where the autonomous system operation needs augmenting in cases of specific difficulty. The communications environment is particularly challenging due the presence of cars and metal frameworks and pillars which can block or reflect signals arbitrarily, and the low profile bogies operating under vehicles which can shield them. The command system research is based on establishing resilience to communications dropout through judicious use of multiple input multiple output systems, the application of spectral diversity to essential command functions and meshing exploration. A series of trials has shown good connectivity between system elements when located under or behind parked vehicles. Within the structure of the command system,
autonomous processes are built in to cope with short term wireless loss and to recover system functions. IV.
AVERT S OFTWARE A RCHITECTURE
The Robot Operating System (ROS) was chosen as the AVERT operating system. Additional to the driver support of the utilized devices, ROS offers more advantages. Since the different devices do not provide direct communication between each other and, moreover, there is no external triggering mechanism for synchronization, ROS undertakes this task using timestamps to the messages published from each device. Furthermore, each sensor data is published as ROS nodes for interprocess communication over the network. The AVERT Command Console is also ROS based and provides the mechanisms for operator planning, scene selection and manual intervention to override the autonomous operation, initially through a stop command and then through the provision of a reversionary teleoperation mode for the AVERT system using familiar joystick controls. A. 3D Reconstruction For the 3D reconstruction the overall set up is comprised of a SICK scanner mounted on the PTU, rotated in order to achieve vertical scanning. SICK’s angular resolution is set to 0.1667◦ and PTU’s angular velocity on the pan axis is 3◦ /sec. The 3D scan is produced by performing a 360◦ scan sweep. The point cloud registration is accomplished into two distinct steps. First, a rough estimation of the transformation matrix is given using the employed Fast Point Feature Histograms (FPFH) [9] features followed by the Iterative Closest Point (ICP) algorithm as a refinement step [10], [11]. FPFH are multi-dimensional features describing the geometry of a point belonging to a 3D point cloud. This feature derivation method is an expansion of PFH [12], the method allows the on-line calculation of those features while retains the discriminative capabilities of PFH, making it suitable for on-line applications. The computational complexity of FPFH is of O(k) in contrast to PFH which is of O(k 2 ). By applying the motion transformation on the respective 3D point clouds we obtain a rough alignment and, as a result, the merged 3D map retains erroneous registrations. Hence, the initially transformed point clouds are considered for the correction of the motion estimation. The most commonly used algorithm for 3D point cloud register refinement is the ICP. The novelty of the AVERT work is that the ICP algorithm considers only the points that belong to specific geometric
surfaces in consecutive time instances. The successive point clouds share great amount of spatial proximity, due to the fact that a coarse alignment occurred during the motion estimation procedure. The benefit from this procedure is twofold: firstly we avoid multiple iterations restricting the rigid body transformation search by one order of magnitude in calculation time and, secondly, we increase the likelihood to achieve an accurate solution as the one shown in Fig 4(a). B. Path Planning The top-down projection of the enhanced map is provided as an input to the D* Lite path planning method. The latter treats the problem as a graph-traversal one and due to the resolution of the map the distance between two nodes in the graph corresponds to 1 cm in the real world. D* Lite is a fast path planning and re-planning algorithm suitable for goal-directed robot navigation in partial known or unknown environments. D* Lite constitutes an extension of Lifelong Planning A* (LPA*) [13], [14] and one of its most significant additions is the occasion where the target position alters during the re-planning episodes. Due to the fact that D* Lite expands LPA*, it also acquires the entire set of attributes that LPA* entails and its expansion capabilities as well. Compared to other methodologies, such as the well known D* [15], D* Lite is simpler, can be rigorously analyzed and extended in various aspects while its efficiency is at least equal to the one of D*. Regarding the simplicity, D* Lite utilizes a single tie-breaking rule as far as it concerns the comparison of the priorities, making the maintenance of the priorities an easy task. The goal-directed navigation is considered to be a special case for the D* Lite algorithm. D* Lite iteratively computes the most efficient route from the current position of the robot and the target one, while the robot is moving against the target with concurrent cost change on the edges of the graph, as shown in Fig. 4(b). The generic nature of the methodology is revealed by the fact that no assumptions regarding the graph are made. More precisely, there is no hypothesis of how the cost of the edges changes, or even the sign of that cost change. Moreover, there are no assumptions about the cost change depending on the current position of the robot, or the knowledge expansion from the robot or the dynamic behavior of the environment. C. Bogies Control system Each half bogie has one processing unit for collecting sensor data, steering the motors and interfacing with the communication network. The communication of the bogie is based on a WiFi network. At software level the robots are abstracted as network nodes. This endows the system with the maximum flexibility to control all of them independently from multiple sites. The main approach is to send all commands for all bogies to one master-bogie, this bogie distributes then all information to its neighbors. The overall system control is implemented on the deployment unit, which should be able to estimate each bogie’s position through a sensor fusion between laser tracking on shape and reflectivity features, wheel odometry and an inertial measurement unit. Several steps have to be accomplished in order to control the bogie for docking at the vehicle’s wheel. After bogie
deployment, a path containing an array of poses (position and orientation) to the front or the side of the blocking car is derived from the overall map. From this point, the bogies needs to move autonomously by searching for the vehicle tires using on-board laser rangers. The bogies firstly position themselves below the center point of each axle of the blocking vehicle, then split in two sections and finally dock at the wheels. There are several states for each bogie, amongst others: joined, split and loaded. Each state causes a separate algorithm for the omnidirectional kinematic control to move in a predicted way. The kinematics of the bogies are mainly described by the movements of the mecanum wheels [16]. Certain angular velocities (ω1 , . . . , ω4 ) on the mecanum wheels lead on to a certain movement of the bogie. Using forward and backward kinematics, the angular velocities for every wheel can be determined originating from a certain bogie movement (position and orientation change) as well as the other way round. One has to differentiate between the movement of a full bogie with four mecanum wheels and a split bogie section with only 2 wheels. In the latter, the kinematics get more complex and are less well determined while the weight distribution is preponderating and needs to be also considered. V.
AVERT S IMULATION
Operational simulations were necessary for defining and evaluating critical operational tactics and configurations. Our simulation was based on the Webots real-time dynamic simulation platform. The operational scenarios were performed on a 150m by 150m 3D virtual world of an indoor parking lot where the deployment unit is equipped with all the aforementioned sensory modalities. The Open Dynamics Engine library was utilized for all the necessary physics features such as mass, friction, communication range and laser accuracy. The omnidirectional motion of the mecanum wheels of the bogies was also modeled for testing the accuracy of direction and bogie movement. A bidirectional communication between the bogies node and the deployment unit node was also implemented. The Pioneer 3-AT mobile robot was used as our IED towing robot for the deployment unit. The simulation environment was particularly useful because it allowed us to perform fast, automatic sensor data collection and analysis over various parameter sets in extensively large operational fields such as the 22500m2 parking lots. End user training and tactical familiarization will be also performed in this environment. VI.
AVERT F IELD R ESULTS
The AVERT system was deployed in a typical indoor parking lot at the premises of Zurich University of Applied Sciences, Switzerland, for preliminary field testing. The bogie prototypes as shown in Fig. 5, were tested in their capacity of attaching and maneuvering accurately the load of a car of typical mass. Successful trials were performed reporting a full vehicle extraction between the initial position until the extraction point with a total covered distance of 20 meters, as shown in Fig. 6. The current configuration of the bogies allows small obstacle
Fig. 5.
Fig. 6. AVERT in action, where the blocking car is extracted through the attached bogies.
AVERT bogie prototype.
overpassing without orientation misalignment. This allows a wider choice of paths to be considered without the application of obstacle avoidance. For the 3D registration and path planning accurate results were reported with the scanning process having a standard execution time based on scan rate while the registration procedure using the FPFH features was accomplished in a significantly lower time than the scan. The refinement procedure of the ICP reported a very short execution time of which the path planning algorithm presented a very small proportion to achieve good sub-second real time performance. VII.
C ONCLUSION
The paper presented the overall framework and on-going research activity of the EU project AVERT focused on introducing a novel competence to security related forces to rapidly deploy, extract and remove blocking vehicles from vulnerable positions. The project targets specifically at semi-autonomous vehicle removal from restricted places with fine handling, in any direction to a safer disposal point so as to reduce or eliminate collateral damages to infrastructure and personnel. While the robotic support and teleoperation methods have been used for some time by the EOD community for EOD robots, the AVERT concept will provide extended capabilities in mapping, planning and situational awareness as well obstacle removal. This will have also a potential impact on the current understanding and training for human-robot interaction by EOD operators.
[2]
[3] [4]
[5]
[6]
[7]
[8]
[9]
[10]
[11] [12]
ACKNOWLEDGMENT This work was fully supported by the E.C. under the FP7 research project for The Autonomous Vehicle Emergency Recovery Tool to provide a robot path planning and navigation tool, “AVERT”, FP7-SEC-2011-1-285092.
[13] [14]
[15]
R EFERENCES [1]
F. Matsuno and S. Tadokoro, “Rescue robots and systems in japan,” in Proc. IEEE Int. Conf. Rob. Biomimetics, 2004, pp. 12–20.
[16]
C. Beltr´an-Gonz´alez, A. Gasteratos, A. Amanatiadis, D. Chrysostomou, R. Guzman, A. T´oth, L. Szollosi, A. Juh´asz, and P. Galambos, “Methods and techniques for intelligent navigation and manipulation for bomb disposal and rescue operations,” in IEEE International Workshop on Safety, Security and Rescue Robotics, 2007, pp. 1–6. B. Siciliano and O. Khatib, Springer handbook of robotics. SpringerVerlag New York Inc, 2008. J. Burke, R. Murphy, M. Coovert, and D. Riddle, “Moonlight in miami: Field study of human-robot interaction in the context of an urban search and rescue disaster response training exercise,” Human– Computer Interaction, vol. 19, no. 1-2, pp. 85–116, 2004. J. Scholtz, J. Young, J. Drury, and H. Yanco, “Evaluation of humanrobot interaction awareness in search and rescue,” in Proc. IEEE Int. Conf. Robot. Autom., vol. 3, 2004, pp. 2327–2332. A. Amanatiadis, S. A. Chatzichristofis, K. Charalampous, L. Doitsidis, E. B. Kosmatopoulos, P. Tsalides, A. Gasteratos, and S. I. Roumeliotis, “A multi-objective exploration strategy for mobile robots under operational constraints,” IEEE Access, vol. 1, pp. 691–702, 2013. C. Fischer and H. Gellersen, “Location and navigation support for emergency responders: a survey,” IEEE Pervasive Comput., vol. 9, no. 1, pp. 38–47, 2010. A. Amanatiadis, A. Gasteratos, C. Georgoulas, L. Kotoulas, and I. Andreadis, “Development of a stereo vision system for remotely operated robots: A control and video streaming architecture,” in Proc. IEEE Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems, 2008, pp. 14–19. R. B. Rusu, N. Blodow, and M. Beetz, “Fast point feature histograms (fpfh) for 3d registration,” in Proc. IEEE Int. Conf. Robot. Autom., 2009, pp. 3212–3217. P. Besl and N. McKay, “A method for registration of 3-d shapes,” IEEE IEEE Trans. Pattern Anal. Mach. Intell., vol. 14, no. 2, pp. 239–256, 1992. S. Rusinkiewicz and M. Levoy, “Efficient variants of the icp algorithm,” in 3DIM’01, 2001, pp. 145–152. R. B. Rusu, Z. C. Marton, N. Blodow, M. Dolha, and M. Beetz, “Towards 3d point cloud based object maps for household environments,” Robotics and Autonomous Systems, vol. 56, no. 11, pp. 927–941, 2008. S. Koenig and M. Likhachev, “Incremental a*,” in NIPS, 2001, pp. 1539–1546. K. Charalampous, A. Amanatiadis, and A. Gasteratos, “Efficient robot path planning in the presence of dynamically expanding obstacles,” in Cellular Automata, 2012, pp. 330–339. A. Stentz, “The focused d* algorithm for real-time replanning,” in IJCAI, vol. 95, 1995, pp. 1652–1659. R. L. Williams, B. E. Carter, P. Gallina, G. Rosati et al., “Dynamic model with slip for wheeled omnidirectional robots,” IEEE Trans. Robot. Autom., vol. 18, no. 3, pp. 285–293, 2002.
