supported by a vast online community providing open-source autopilot software of varying capability, e.g., [29, 30]. The sheer number of developers and users ...
Important Notice: This is the authors’ pre-publication version. This paper does not include changes and revisions arising from the peer review and publishing processes. The final definitive copy, which should be used for all referencing, is published at https://doi.org/10.1016/j.paerosci.2017.03.002
1
Enabling Technologies for Autonomous MAV Operations Elbanhawi M.1, 2, Mohamed A.1, Clothier R. 1, Palmer J.L. 3, Simic M. 1, and Watkins S. 1 1 School
of Engineering, RMIT University, Melbourne, Victoria 3001 2
3
MEMKO, Melbourne, Victoria 3000
Defence Science and Technology Group, Fishermans Bend, Victoria 3207
Abstract The utility of micro air vehicles (MAVs) has expanded significantly in the last decade, and there are now numerous commercial systems available at relatively low cost. This expansion has arisen mainly due to the miniaturisation of flight-control systems and advances in energy storage and propulsion technologies. Several emerging applications involve routine operation of MAVs in complex urban environments, such as parcel delivery, communications relay, and environmental monitoring. However, MAVs currently rely on one or more operators-inthe-loop; and, whilst desirable, full autonomous operation has not yet been achieved. In this review paper, autonomous MAV operation in complex environments is explored with conceptualisation for future MAV operation in urban environments. Limitations of current technologies are systematically examined through consideration of the state-of-the-art and future trends. The main limitations challenging the realisation of fully autonomous MAVs are mainly attributed to: computational power, communication and energy storage. These limitations lead to poor sensing and planning capabilities, which are essential components of autonomous MAVs. Possible solutions are explored with goal of enabling MAVs to reliably operate autonomously in urban environments.
Nomenclature AAS
Autonomous aircraft system(s)
AFLUS
Autonomy levels for unmanned systems
ARR
Aircraft-response ratio
ASIC
Application-specific integrated circuit
ATM
Air traffic management
BLoS
Beyond line-of-sight
CALM
Communication access for land mobiles
CCL
Command and control link
CMOS
Complementary metal–oxide–semiconductor
CNPC
Command and non-payload communication
DARPA
(U.S.) Defence Advanced Research Projects Agency
DSRC
Dedicated short-range communications
e
Specific energy
EHF
Extremely high frequency
EO
Electro-optical
FCS
Flight-control system
2
FoV
Field of view
FPGA
Field-programmable gated array
GCS
Ground control station
g
Acceleration due to gravity
GNC
Guidance, navigation, and control
GNSS
Global navigation satellite system
GPS
Global Positioning System
ICAO
International Civil Aviation Organization
INU
Inertial navigation unit
IR
Infrared
ISM
Industrial, scientific, and medical
ISR
Intelligence, surveillance, and reconnaissance
ITU
International Telecommunication Union
LB
Logic blocks
LED
Light-emitting diode
LEDDAR
Light-emitting diode detection and ranging
L/D
Lift-to-drag ratio
LIDAR
Light detection and ranging
LiPo
Lithium polymer
LoA
Level of autonomy
LoS
Line of sight
MAV
Micro air vehicle
MEMS
Micro-electro-mechanical system(s)
OEM
Original equipment manufacturer
PML
Payload-management link
PRM
Probabilistic roadmap
RP
Remote pilot
RPAS
Remotely piloted aircraft system(s)
ROACH
Radar-on-a-Chip
RRT
Rapidly exploring random trees
SLAM
Simultaneous localisation and mapping
3
SHF
Super-high frequency
S/N
Signal-to-noise ratio
SPA
Sense–plan–act
SPAD
Single-photon avalanche-diodes
SWaP
Size, weight, and power
T
Time
ToF
Time-of-flight (sensor)
UAS
Unmanned aircraft system(s)
UHF
Ultra-high frequency
WLAN
Wireless local area network
WIMAX
Worldwide Interoperability for Microwave Access
V
Velocity
η
Efficiency
1
The Need for Autonomy
The original impetus for micro air vehicles (MAVs) by the U.S. Department of Defense was the development of an aircraft weighing less than 90 g with a maximum dimension of ~0.15 m [1]. Reduction of vehicle dimensions and mass are key features affecting their transportability and allowing rapid field deployment. The primary flight requirements were effective sensor placement for intelligence, surveillance, and reconnaissance (ISR), detection, and communications missions. Furthermore, operation in a variety of complex environments and the capability to conduct missions in all weather conditions is desirable. Although not all of the original requirements have materialised and seemed optimistic for the 1990's, they are still being pursued. Today MAVs are part of the rapidly growing commercial “drone” market. These vehicles are ideally suited to close-range ISR, sensing and communications applications. Such applications typically occur within challenging and complex environments. Many MAVs require human operators to control them through line-of-sight (LoS) communication links. Typically, a MAV remote crew comprises a minimum of two personnel. The first is the remote pilot (RP), who is responsible for the safe flight of the MAV; whilst the second crew member typically manages the payload and mission aspects [2]. More complex missions, such as the use of MAVs for disaster response or in complex environments, may require additional remote crew members [3]. Manual MAV operators are limited by human factors, such as manual control capacity and bandwidth. Human manual control has been shown to underperform inertial-based flight controllers in stabilisation tasks [4]. Recent studies have demonstrated that manual control of MAVs in cluttered and dynamic environments is a challenging task, even for experienced operators [5] [6].The limited task-load capacity of human operators restricts MAV operations because the introduction of tasks such as payload management and comprehension of the acquired data require significant human resources. Controlling, guiding, and managing a swarm of MAVs would be a yet more challenging task. However, recent research indicates [7] that operations of remotely piloted aircraft systems (RPAS) are limited by technological failures, rather than by operator errors. To minimise operator workload, MAVs must be capable of independent decision making and operation. The study by Young et al. [8] highlighted the significance of autonomy for unmanned aircraft system (UAS). Enabling autonomy is expected to reduce the number of staff (e.g., pilots, engineers, mission planners, and payload operators), increase mission success, increase information collection, reduce mission cost, and reduce the risks caused by human operator performance (e.g., response time, work overload, and fatigue). Ideally, autonomous MAVs would also allow a single operator to define a cooperative mission and to monitor the performance of a MAV swarm. Despite the significant advances in artificial intelligence, robotics, sensing, and computational performance of hardware in the last decade, development of autonomous MAVs remains a challenge due to their unique constraints of size, weight, and power (SWaP), along with operational requirements and environmental challenges. The subsequent consequences limiting autonomy are described in Figure 1.
4
MAVs are intended primarily for urban environments, including indoor operations. Such missions require that MAVs operate within highly dynamic, obstacle-rich environments. A number of limitations arise within the urban environment. Robust sensors are needed to detect and track dynamic and static obstacles (e.g., buildings, trees, traffic lights, people, cars, and others MAVs). In addition, urban environments are often characterised by unreliable or non-existent global navigation satellite system (GNSS) signals. Hence, environmental sensing solutions (e.g., vision, laser, and inertial sensors) are needed to localise the aircraft. Sensors of an appropriate scale have their own associated challenges. Computationally efficient planning algorithms are needed to define actions that ensure mission success and MAV safety due to the inherently highly dimensional and constrained state space. This environment can pose communication issues due to interference, lack of infrastructure, and spectrum unavailability. In the presence of wind, large obstacles such as buildings in urban environments increase turbulence levels, which will challenge traditional attitude control and require novel solutions for flight path planning [9]. We envisage a future of small, agile, and highly autonomous MAVs that can provide services to society in the form of swarm sensing (Figure 2). The potential applications of MAVs are only limited by designers’ and users’ creativity; however, to realise the full potential of MAVs, scientists and engineers must overcome various contemporary technological challenges. The specific technical limitations are explored in this paper. The primary aim of the paper is to identify promising technologies and research trends for enabling autonomous MAV operations, i.e., enhancing their level of autonomy (LoA). Hence, we identify the unique constraints of MAVs and current technological limitations of their autonomous operation. An introduction to autonomous MAV operation is presented in Section 1. An analysis is presented in Section 2 of current limitations and the future trends of technologies related to autonomous MAVs. Technological limitations are classified into avionics, computational processing, communication, and energy storage. Hardware limitations are also considered, including identification of the consequences of such limitations on MAV autonomy (Section 2) and how requirements for autonomous MAV operations are affected by the planning and environmental sensing capabilities of autonomous MAVs (Sections 3 and 4, respectively).
5
MAV Unique Constraints
Technological, and Physical Limitations
Hardware Limitations
Consequences on Autonomy
Technological Limitations
Size and Weight Limitations
Miniature Size
Poor Computational Performance Storage Limitations
Poor Cooling
Limited Energy Storage
Low Power
Poor State Estimation
Low Mass / Inertia Non-Linear Flow Phenomena
High Perturbation Rates
Low Autonomy Higher Relative Turbulence Operational Requirements
Slow & Low Level Flying Cluttered Obstacles
Vibration, Routine Impacts & Noise
Environmental Challenges
Poor Avionics/ Sensor Performance
Failure to Capture Vehicular / Environmental Dynamics
Clouds, Humidity, Rain, Heat & Sunlight
Dynamic Obstacles
Figure 1. Unique MAV constraints and their influence on the need for autonomy in a single MAV
6
Figure 2. Future exploitation of MAV swarms for urban operations
1.1
What is Autonomy?
A contextually appropriate and measurable evaluation of the LoA is critical for regulators to evaluate the safety of an autonomous MAV [10]. Hence, it is essential to survey the definitions of autonomous aircraft for the purposes of discussion in this paper (note that this remains an active area of research). The Oxford English Dictionary [11] defines autonomy as the “right or condition of self-government”. A review of various definitions and scales of autonomy that have been applied to MAVs and UAS is provided by Clothier et al. [12]. The definitions of autonomy in that review were separated into two categories based on “independence” and “complexity”. The first category describes autonomy as the independence of the UAS from the RP. The second category relates increasing autonomy to increasing complexity to resemble human intelligence. Several frameworks attempt to define UAS autonomy based on independence [13], and the International Civil Aviation Organization (ICAO) [14] referred to an Autonomous Aircraft System (AAS) simply as an aircraft that operates without human intervention. Clough [15] argues for the separation of automated, autonomous, and intelligent behaviours. Clough defined an automatic (remotely piloted) system as one that obeys human commands, but is devoid of any decision-making capabilities. Clough continued to describe autonomous systems as ones that can perform diverse levels of decision making to achieve mission requirements. 1.1.1 Evaluating Autonomy Researchers have proposed scales to measure the independence of a system in order to evaluate its autonomy. Parasuraman et al. [16] proposed ten distinct LoA, with the lowest being human controlled and the highest, a fully independent system. Similarly, ten LoA for unmanned aircraft were proposed in the U.S. Department of Defense UAS Roadmap [17], with the highest corresponding to a “fully independent swarm” of aircraft. Similarly, Clough [15] defines the highest level of artificially intelligent behaviour for a UAS as an intelligent system that is capable of setting and executing its own mission goals. In the context of long-endurance UAS, five LoA were defined, based on the percentage of human interference throughout the mission [8]. The Autonomy Levels for Unmanned Systems (ALFUS) framework, developed by Huang [18], proposes that UAS autonomy be evaluated in relation to three orthogonal dimensions: (1) mission complexity, (2) environmental complexity, and (3) human independence. The framework combines the two definitions of autonomy (i.e., complexity and independence) that were identified by Clothier et al. [12]. In Figure 3, ALFUS evaluation is illustrated in an example with two different types of missions having varying LoA. In accordance with the ALFUS framework, a mission that requires a MAV to follow a set of sub-goals in a controlled laboratory environment (the red points in Figure 3) is considered less autonomous than another mission requiring navigation and coverage of an unknown outdoor environment (the blue points). Similar multidimensional scales of independent variables have been proposed for scaling autonomy by Wang and Liu [19] and by Insaurralde and Lane [20].
7
Figure 3. ALFUS framework, evaluating two different missions (shown in red and blue) of varying autonomy levels. Adapted from [18]. The frameworks proposed by Parasuraman et al. [16] and Huang [18] are widely cited;. however, they are not applicable to all situations. ALFUS requires further work in the specification of objective measures of its component dimensions (e.g., measurements of “environmental complexity”). The resolution of the levels of each axis in ALFUS is not defined, increasing the reliance of the assessor’s subjective judgment and bias. Frameworks that rely on multidimensional scaling do not provide guidance on objectively measuring the scales or on combining the scales to evaluate overall autonomy. Empirical formulas have been introduced as measures of autonomy. Intelligence and elegance were proposed as two measures by Young et al. [8]: Intelligence = mission success x environmental and operational complexity / number of UAS Elegance = Intelligence / (number of UAS x autonomous-system complexity)
(1.1) (1.2)
environmental and operational complexity = inverse of target frequency x degree of interaction x degree of resistance x degree of inaccessibility x number of sensors x required control-input rate x number of control actuators x robot mobility degrees of freedom
(1.3)
autonomous-system complexity = mean processor instruction or operation speed x size of system dynamic memory x number of processors x number of conditional (heuristic) rules x number of robotic behaviours x number of state-space variables x lines of software code
(1.4)
8
A combination of the duration, amount, and quality of RP control was used to evaluate ‘autonomy’ by Doboli et al. [21]: autonomy = ∫ ∫ ∫ (human effort) d(performance) d(area) d(time);
(2)
and a measure of the degree of human dependence was used to evaluate autonomy by Curtin et al. [22]:
autonomy = coefficient (
control bits total message size
)-i (
contact time total mission time
)-j
,
(3)
where the coefficient and i,j indices are evaluated empirically. Finally, we summarise the highest levels of MAV autonomy in three behaviours, based on the reviewed work. An autonomous MAV will possess: 1) inference: to infer situational awareness from its sensory measurements 2) reasoning: to reason and define its mission based on abstract, human-defined goals 3) unsupervised learning: to adapt and learn its own control strategies without human supervision. Research into the meaning of autonomy and its measurement is ongoing. Clothier et al. [12] concluded that there was no consensus on the concept of autonomy or on how it should be measured and that the most appropriate definition will depend on the context of its use. Beyond the definition of autonomy, the identification of acceptable LoA to maximise mission success requires further investigation. 1.1.2 Regulatory Definition of Autonomy There is an evident terminology clash in the literature and the media for autonomous and unmanned aircraft. For the purposes of this paper, we resort to the regulatory definition of UAS by ICAO [14], which separates UAS into RPAS and AAS. The overarching definition for MAVs falls under the ICAO definition for UAS, i.e., “An aircraft (or aircraft-system) that is flown from a remote location without a pilot located in the aircraft itself”. This definition encompasses a wide scale of MAV autonomy levels (Figure 3), at one end of which lies intelligent swarms of UAS. On the other extreme are remotely controlled MAVs. ICAO defines RPAS as “a form of UAS which is non-autonomous in its capacities, the aircraft being subject to direct pilot control at all stages of flight despite operating ‘remotely’ from that pilot”. In case of RPAS, the human operator provides all the necessary commands and decision making. Between these extremes lies a spectrum of varying levels of intelligence and human independence, which are areas of continuing research. Following the ICAO definition, autonomous MAVs are capable of executing human-defined missions without operator intervention. We consider an autonomous MAV’s operation during a human-defined mission, with or without human monitoring. This may include coordinated action with other MAVs (i.e., a swarm). We assume that the LoA is maintained for all phases of the mission and functions of the MAV, i.e., we do not consider adaptive autonomous behaviour). This abstract definition is appropriate for the scope of this paper, because the definition of autonomy is not our subject, but rather the driving technologies to increase LoA. 1.2
Do Fully Autonomous MAVs Exist?
Fully autonomous MAVs are yet to be realised. In 2012, a competition held by the U.S. Defense Advanced Research Projects Agency (DARPA) showed that no MAV was capable of operating beyond LoS (BLoS) for a perch-and-stare mission due to various technological limitations, evidence that much work to enhance the performance of MAVs is still needed [23]. As discussed by Prior et al. [23], the ability to operate without LoS communications between the MAV and the RP and resilience to outages of the GNSS were the main factors resulting in mission failure. Until 2010, there were no examples of autonomous swarms collaborating independently of external input [24]. Limited usage of a single UAS has been reported for search-and-rescue operations with a ground or water robot [25, 26]. More recently progress was reported by Scaramuzza et al. [27], who reported the deployment of multiple small UAS to map an abandoned environment. Despite their significant advancement, the operation of the UAS was limited to a smallscale mission (of 350-m extent) in an environment completely sparse of structures. They also relied on off-board processing of some sensor information. The limitations of this approach are described in Section 2, where communications are discussed. Nonetheless, this represents a promising step towards future urban operations. Higher LoA are being achieved by larger UAS, which have lesser SWaP restrictions than MAVs (Figure 1). An example is the autonomous helicopter developed in the Smart Skies Project [28]. The mission required the UAS to navigate beyond the visual range of the RP in an unknown outdoor environment to collect photographs of a target location. On
9
the return leg of the mission, the UAS was required to detect and avoid a simulated conflict with another low-flying aircraft. All of this was achieved without interaction from the RP. If we adopt the ALFUS framework, then the helicopter UAS would be rated “high” along each of the three dimensions. It is therefore apparent that miniaturisation imposes various challenges for fully autonomous MAV operations. In an effort to explore the underlying limitations preventing fully autonomous MAV operations, a generalised sense–plan–act (SPA) cycle is used to describe the generic functions of an autonomous system, as shown in Figure 5. Over the past decade, robotics has evolved from the realm of automation to higher LoA. In the discipline of artificial intelligence, autonomous robots (agents) operate through an SPA cycle, described as: • • •
Sense: Utilising a wide array of sensors for the purpose of attitude, location, and velocity measurements amongst clutter Plan: Systematic policy of generating a feasible, continuous set of actions from the current state to a desired state to achieve an overall goal Act: Executing actions through the implementation of control laws.
The ‘act’ component of the cycle describes the function of micro-sized autopilots and flight-management systems (the on-board “virtual pilot”). Low-cost micro-sized autopilots are now readily available, predominantly due to the falling cost and increasing performance of micro-electro-mechanical systems (MEMS) and micro-controllers. These systems are supported by a vast online community providing open-source autopilot software of varying capability, e.g., [29, 30]. The sheer number of developers and users has led to a rapid increase in the reliability and performance of these costeffective autopilot systems. The market has driven the maturity of the technologies related to the ‘act’ component of the SPA cycle. The ‘sense’ and ‘plan’ components of the cycle are still underdeveloped, as evidenced by the lack of commercially available autonomous systems. ‘Plan’ and ‘sense’ aspects are investigated in Sections 3 and 4 of this paper, respectively.
Figure 4. The trends driving increasing LoA
1.3
Existing Reviews of MAV Autonomy
Other surveys that consider technological challenges of UAS complement this paper. Kumar and Michael [31] focussed on the challenges of the ‘act’ component of the SPA cycle, along with controlling multi-rotor MAVs. They surveyed relevant planning and sensing methods; however, a discussion on the particular sensing or planning requirements for realistic MAV deployment was not provided. Kendoul [32] presented a detailed survey on the guidance, navigation, and control (GNC) systems associated with an autonomous rotary-wing UAS. The complexity of the UAS ‘plan’ component, in comparison with typical robotic planning, was also highlighted by Goerzen et al. [33]. Neither Kendoul [32], nor Goerzen et al. [33], place particular emphasis on the limitations specific to MAVs. A review by Petricca et al. [34] of rotary-wing MAVs focusing on airframe designs, propulsion systems, and energy sources did not consider autonomy. Therefore, in this paper we investigate the sensing and planning requirements for real-world operation of MAVs, independent of any human control. A sensor-taxonomy framework from the perspective of autonomous MAVs is also proposed.
10
Figure 5. An overview of the autonomy sense–plan–act cycle for remotely piloted, autonomous, and intelligent UAS and swarms, using ICAO terminology
11
2
MAV Autonomy Limitations
In Section 1, we identified operational requirements and miniature size as unique constraints of autonomous MAVs in comparison with large-scale UAS. In this section we identify the consequences of existing technological limitations on MAV autonomy. Autonomous navigation requires the aircraft to be capable of processing all incoming data to execute the SPA cycle at a relatively high rate. There are a number of factors that restrict increasing MAV LoA, as illustrated in Figure 6. These limitations largely influence the ‘sense’ and ‘plan’ components of the SPA cycle. A small number of MAV research platforms that utilise custom-designed flight computers were successful in performing tasks entirely on-board, such as vision-based state estimation [35, 36] and indoor navigation of partially known regions [37]. Another approach is transferring a portion of the data to an off-board computer in the ground control station (GCS) for demanding tasks. This approach relies on distributing tasks amongst limited on-board computers, carried by the aircraft, and more powerful off-board computers in the GCS. However, there are technical challenges to both strategies, namely on-board and off-board processing, as the data input and the task complexity increase. On-board processing is limited by the associated increase in energy consumption, limited computational power, and increased costs of smaller processing units. In contrast, off-board processing is challenged by the limitations of communication technologies (e.g., limited data rates, bandwidths, latency, range, and power consumption). The increase in the quantity of high-fidelity data transfer would deplete the limited on-board energy resources; however, as is explored later, communication capacity remains limited. Selective communication between a swarm of aircraft and the central station can also moderate computational tasks. It is clear that limitations of energy storage and power generation directly limit both on-board computational capacity and communication bandwidth. The payload of the aircraft limits onboard energy storage and computational resources, which in turn influence the system’s autonomy (Fig. 5). The following subsections therefore explore common limitations (i.e., avionics and communications). This is then followed by exploration of the specific limitations associated with the sensing and planning components (Sections 4 and 3).
Planning limitations
Robust sensor technology
Navigation (GNSS-denied environments)
Attitude control (Turbulent environments)
Obstacle detection (Ground & air)
Common Limitations
Sensing limitations
High implementation rate
Safe navigation (Cluttered urban environments / dynamic surroundings)
Kinodynamic planning (Highly dimensional state-space)
Common limitations Hardware performance
Size & weight
Responsiveness
Computational requirements
Communications
Range
Available bandwidth
Required data rate
Latency
Power requirements (Energy-storage limitations)
Figure 6. Technical limitations on MAV autonomy
12
Signal reliability
Infrastructure
2.1
Avionics
2.1.1 Size and Weight By definition, MAVs are miniaturised (refer to the common challenge in Figure 6); thus, real-time execution of the SPA cycle is primarily limited by SWaP constraints. SPA complexity and execution requirements are expected to increase with autonomy. These limitations are categorised into avionics performance (Section 2.1), communication (Section 2.2), and energy storage and power generation (Section 2.3). Evidently, such problems are less apparent for largescale UAS, which contain more physical space to accommodate processors and energy storage. This highlights a design trade-off that is particular to MAVs. The size and weight limitations of MAVs are reflected in the requirements of the on-board avionics, specifically the flight-control system (FCS). Current commercially available computers of suitable size for MAV usage often do not deliver sufficient performance for complete on-board processing [27]. The exponential increase in transistor density and decrease in size are expected to continue for several more years [38], as shown in Figure 7. The continued miniaturisation of hardware will increase the availability of smaller, higher-performance computing hardware suitable for implementation on-board MAVs.
Figure 7. Transistor density (number of transistors/mm2) and minimum feature size (mm) in microprocessor units. Adapted from [38].
2.1.2 Computational Performance SPA cycle tasks require significant computational power and are challenging for on-board processors. Researchers will overcome the computational complexity of autonomous flight through optimised perception solutions and adaptable real-time planning algorithms. Novel sensing solutions may provide a route towards reducing the computational load. However, computing performance remains a significant challenge for MAV autonomous operation. The size, cost, and energy resources available place stringent restrictions on the type of processors that can be carried on-board MAVs. 2.1.2.1 Processors The performance of on-board processing systems directly relates to task complexity and subsequently to the autonomous capabilities of the aircraft. In Section 1.1, we argued that achieving autonomy requires a level of independence from RP oversight. This necessitates that the aircraft perform a significant portion of data analysis and decision making on-board, through the FCS. Inspecting Figure 8 and Figure 9, one may observe that processing speeds and computing power have been increasing and processor costs are significantly decreasing. Note that in Figure 8, computational speeds are shown for single-core processors. Despite these significant improvements, current UAS applications are limited to minimal collaboration between a small number of aircraft and to reactive path planning [24]. It is clear that technological advancements in computing have not been utilised to their full potential in autonomous systems [27]. It is safe to assume that computational resources will not limit the future of unmanned systems, rather that they are limited only by the efficiency of utilisation of those resources. From 2006, multi-core processing and on-board graphics-processing units were introduced, whilst improvements in processor speeds remained relatively constant. These technological developments have already begun to contribute to accelerating planning and mapping algorithms [39, 40].
13
Figure 8. Desktop-computer processing power (1999–2014). Displayed speeds are for the highest-specification model released in each year [41] and [17]. Adapted from ….???
Figure 9. Inflation-adjusted Intel processor prices relative to processing speed (2000–2014) [41-44]. Adapted from ….???
2.1.2.2 Field-Programmable Gated Arrays (FPGAs) Field-programmable gated arrays (FPGAs) are chips that can be physically rewired when reprogrammed after their fabrication, unlike the aforementioned processors, i.e., application-specific integrated circuits (ASICs) that run software-defined instructions. Using artificial neural networks [45], FPGAs have been used to mimic human decision making, which is a desirable function for autonomous MAVs. The benefits of FPGAs over ASICs are rapid-prototyping capabilities, dedicated hardware, and the elimination of circuit redesign. FPGAs are increasingly becoming key technologies in digital-circuit design, particularly for low-volume products [46]. The significant cost reduction (by more than a factor of 100 since 1990) has exponentially increased the use of FPGAs in products and design projects, mostly for digital-signal processing and custom computing [47]. Monmasson et al. [48] reviewed the advantages of FPGAs for control applications. These chips allowed for the development of smart sensors (e.g., self-diagnosis and sensor networks) and have been used in computer-vision and image-processing applications [49]. There are several challenges associated with FPGAs, particularly power consumption, complexity, time delay, and size. FPGAs consist of logic blocks (LBs). There exists a design trade-off in the device’s design between its speed, area, and power consumption based on the LB size and its functionality [50], as illustrated in Figure 10. An experiment conducted between FPGAs and ASICs using the same logic elements concluded that an FPGA is on average approximately 35 times larger and between 3.4 to 4.6 times slower [51]. Estimating the timing delay of FPGAs for complex circuits is another challenge, as the majority of the time delay is caused by the interconnectivity between different LBs [46].The aforementioned challenges of FPGAs limits their utility for autonomous MAVs. The current state of technology suggests that these chips will be limited to sensing, communications, and digital-signal processing for now. 14
Figure 10. FPGA-design trade-off
2.1.3 System Responsiveness Autonomous deployment requires reliable and real-time execution of the SPA cycle. The MAV must be capable of sensing its surrounding environment and processing that information to react and update its current state within suitable timeframes based on its flight speed. Robust autonomous system deployment has been previously possible for large-scale robots in relatively structured environments [52]. Kelly and Stentz [53] define a system-response ratio, which is the ratio of the response distance to sensor look-ahead distance (DLook-Ahead). An adapted measure of responsiveness for aircraft, referred to as aircraft-response ratio (ARR), is defined as: ARR =
𝑉𝑅𝑒𝑙 x TRes
DLook-Ahead
.
(4)
To sustain safe navigation, this ratio must be kept above unity. The response distance is governed by the relative velocity at which a MAV closes with static and dynamic obstacles (VRel) and its response time (TRes). Limitations on computation restrict the response time of the system to process the incoming data (Tsense), to update the environment and its own state (Tupdate), to modify the current plan (Tplan), and to react accordingly by executing the plan (Treact); thus, TRes = Tsense + Tupdate + Tplan + Treact.
(5)
The component Treact must include the time required to manoeuvre the MAV (within constraints) to ensure a minimum safe clearance distance. Sensor range and field of view (FoV) pose restrictions on the system’s look-ahead distance. Section 4 uses ARR to illustrate responsiveness of various sensors. This adapted ARR is a relaxed estimate of MAV responsiveness with unimpeded FoV and no manoeuvre constraints within static surroundings. As the assumptions are removed the responsiveness is expected to decrease (refer to Section 3 and Section 4), highlighting the significant challenge of autonomous navigation in urban environments.
Figure 11. Minimum response time to guarantee safe navigation at different speeds. Separate curves are for different lookahead distances.
15
2.2
Communications
Communication is essential for integrating MAVs within civilian airspace. The integration of UAS has been studied extensively [54]. In this Section we discussed MAV-specific challenges. There are two categories of links for a UAS as defined by ICAO [14]: namely, a command and control link (CCL) and a command, control, and communication (C3) link. A CCL (C2) link is ‘the data link between the remotely piloted aircraft and the remote pilot station for the purposes of managing the flight’. It is referred to as the command and non-payload communication. The C3 link includes voice and data transferred between the UAS and the GCS. For the purposes for this paper, we will refer to C2 and C3 links as CCL and payload-management link (PML), respectively. 2.2.1
Communication Links
2.2.2 MAV Communication Purposes MAV communications links generally serve four purposes, as outlined in Figure 12. In addition to the conventional CCL and PML, communications are needed for off-board computation and (typically) for multi-agent swarming. Except for fully autonomous systems, MAVs are required to relay information back to the GCS regarding flight and/or mission parameters. As mentioned in Section 1, as LoA increases, the dependency on GCS communications is expected to decrease. Frew and Brown [55] categorised UAS GCS communications into direct, satellite, cellular and mesh types, as shown in Figure 13. The second purpose is within multi-agent fleets (swarms) that are capable of sharing information, interacting, and flying in formation. This type of communication is required to realise the highest LoA, as illustrated in Figure 5. The third purpose comes as a consequence of limited on-board computation, whereby a number of data processing and decision-making tasks are communicated to the GCS to be executed off-board by groundbased computers using the PML. The final purpose is a requirement for specific applications, which may require the MAV to act as a communication relay station between vehicles or cell towers.
MAV communication purposes
Critical flight/ mission information to operator
Relaying information/ communications from other entities
Multi-agent cooperation (Swarming)
Off-board data processing
Figure 12. Possible communication purposes of MAVs
Figure 13. MAV and GCS communication types [55]
2.2.3 Direct Links from MAV to GCS Several crucial technical challenges arise with reliance on terrestrial communication networksi.e.. Current 16
communication technologies are limited by range, data rate, latency, and antenna SWaP. Several attempts have been made to standardise the data rates required for UAS missions and tasks [56]. Wireless local area networks (WLAN), IEEE 802.11a/b/g/n/ac standards, allow for relatively high data rates, as shown in Figure 14. They have been utilised on a number of research platforms [35]. However, WLAN technologies suffer from limited range (less than 250 m) and latency (~50 ms); therefore, they cannot be fully utilised for BLoS operations, nor to relay critical processes (Figure 15). Another standard, Worldwide Interoperability for Microwave Access (WIMAX), is based on the IEEE 802.16 standard and provides a more promising avenue with higher data rates and range. It is clear that careful analysis is required before selecting a particular communication technology.
Figure 14. Communication data rates (kbits/s) and operation bands (GHz) in comparison with task-specific required data rates [56] [57].
Figure 15. Range (m) and latency (ms) for various communication technologies [57]
Radio-frequency spectrum availability limits UAS communications . These communications are managed and regulated by international bodies and national governments, with various bands for different applications defined by the International Telecommunication Union (ITU) [58], as illustrated for Figure 16. Most bands are illegal for UAS communications; however, Industrial, Scientific, and Medical (ISM) bands are unlicensed bands that can be utilised for UAS applications. ICAO also restricts CCL to the protected aviation spectrum. ITU splits the frequency bands into three regions that cover different portions of the globe. Additional operational restrictions on radio-frequency bands, e.g., antenna power, gain, and bandwidth, are enforced by international and national government bodies. Generally, low-frequency signals can permeate dense and large objects, but have low ranges and require large antenna sizes. For MAVs, a compromise is made amongst miniaturisation, BLoS, and operation in (urban) clutter. To reduce antenna size, higher-frequency bands are desirable. However, high frequencies limit the permeability of the signal through structures. MAV SWaP limitations restrict the distance travelled by the signal, as additional power and larger antennas are needed, as indicated by equation (6). ITU-defined Ultra-High Frequency (UHF) bands (300 MHz – 3 GHz), which are suited for LoS use are mostly occupied by television broadcasting and mobile communications. In the UHF, the 433- and 915-MHz bands, i.e., amateur or license-free bands, are often utilised for CCL. These bands can achieve a LoS range of 35 km. Their use is designated for ITU region 1 and 2 countries, respectively. Super-High Frequency (SHF) bands (3–30 GHz) are dominated by satellite communication, cellular phones, and WLAN and are 17
utilised for relatively long (30 km) point-to-point communications. The SHF band includes two commonly used ISM bands at 2.4 and 5.8 GHz. They are often utilised for direct communication between mobile devices, WLAN, and cordless devices. The 2.4-GHz band strikes a balance between antenna size (and hence cost), signal penetration, and range for domestic use. However, SHF use in urban environments is limited due to its inability to penetrate built structures. Beyond 30 GHz, the Extremely High Frequency (EHF) band is limited by atmospheric attenuation, as it is absorbed by gases in the atmosphere, as shown in Figure 16. EHF, i.e., mmWave, transmissions are limited to short ranges (1 km). Kerczewski et al. [59] estimated that a spectral bandwidth of 34 MHz is needed for LoS CCL and 56 MHz for BLoS operation based on the ICAO criteria [14]. These requirements do not include PML. LoS CCL can be divided between two bands in the UHF range to improve signal reliability. However, spectrum analysis indicates that there is no available bandwidth for BLoS CCL within the protected aviation spectrum [59].
Figure 16. Radiofrequency spectrum bands and general purposes
Increasing data rates and range requirements pose another challenge. They lead to escalations in consumption of scarce on-board energy and therefore reduce the integrity of the data [60]. For basic operations, such as indoor obstacle avoidance, power consumption for communications is often marginalised [61]. Existing data-rate estimates are mostly based on RPAS and do not consider autonomous operation, e.g., [62]. Transmitted signal power through free-space [63] is directly proportional to the square of the distance between the transmitting and receiving antennas to maintain a fixed received-signal power. Theoretically lossless and isotropic antennas operating in free space are described by: Transmitted Power Recieved Power
=
Area of Transmitter × Area of Reciever Wavelength2 ×Distance2
.
(6)
Transferring data to the GCS for off-board processing may not be an ideal solution as the autonomous capabilities (and thus computational requirements) of the MAV increase. Subsequently it seems sensible that high data-rate communications should be minimised whereby only the most critical information should be sent back to the human operator for high-level decision making. An example of this concept can be implemented for applications/missions that can benefit by replacing the continuous live video feed with only a few high-resolution images of the desired target. This concept can be achieved through escalation of MAV autonomy levels. Whilst there is a clear need for a communication infrastructure that can utilise existing cellular networks, higher levels of autonomy in relation to intelligent data processing on-board the MAV will be required despite the continuing advancements in the performance of ground-based terrestrial data networks. This places a higher demand for on-board processing hardware. 2.2.4 Cellular Links Cellular networks provide another possible medium for BLoS communication. They appear to be the most promising solution for long-distance, high-bandwidth communication. Telecommunications providers are investigating the use of 4G networks for relaying live video and sensor data for UAS operation at high altitudes [64]. There are common challenges in the use of cellular networks such as 2, 2.5G, 3G, and 4G. The networks are designed and optimised for ground-based user terminals. Cell-tower antennas are generally mounted to improve coverage at altitudes below 200 ft. Whilst the availability of modern networks is good, coverage is generally limited in rural areas. Indeed, cellular coverage is likely to be insufficiently robust for UAS operations in the U.S. [65]. Goddemeier et al. [66] analysed the suitability of cellular networks for communication between the MAV and GCS. They developed a model to address the lack of LoS communication especially in cluttered urban environments and 18
concluded that the cellular networks are limited to an altitude of 500 m in urban environments, which is typical for MAVs. Three zones were proposed, based on MAV altitude (up to 30 m, up to 60 m, and beyond 60 m), as shown in Figure 17. Altitudes up to 30 m were also shown to be dominated by masking from buildings and trees, whereas altitudes above 60 m were governed by ground-reflected signals (multi-path interference). A cellular network re-uses frequencies and codes in non-adjacent cells. This can be problematic for airborne devices, which can often establish connections with many non-adjacent towers. The higher than normal effective antenna altitude of the MAV also allows it to establish strong connections with very distant towers, though it may be unable to establish data exchange with the tower due to time-based “range gates” on connections. Thus, a strong connection does not always result in a usable link. Connection-management algorithms within modems specifically designed for ground-based users may not take these effects into consideration. Coverage and performance of the terrestrial cellular networks is also dynamic. Services degrade “gracefully” as consumer load increases; however, the tower service area will effectively reduce as load increases. Additionally, there is an asymmetry between upload and download rates, which is unsuitable for UAS, because they have a high upload rate; whereas the network is geared towards higher download rates. Fifth-generation, mmWave (5G cellular) represents a promising prospect for cellular communications; as it is characterised by high data-rate capacity and low latency [67]. Currently, 5G cellular, lacks an infrastructure compared with existing cellular networks.
Figure 17. Altitude influence on MAV-to-GCS cellular communication signals. Adapted from [66].
2.2.5 Mesh Links Additional communications infrastructure issues must also be considered, including bandwidth availability, link security, and reliability. Eventually, dedicated architectures similar to the Dedicated Short Range Communications (DSRC) [68] and Communications Access for Land Mobiles (CALM) [69] frameworks, which have been developed for intelligent ground vehicles, may provide alternative communication solutions for UAS. GCS data transfer for off-board processing may not be a suitable solution as the autonomous capabilities of MAVs increase. Eventually, standard, certifiable communications links will be needed for autonomous MAV operation. 2.2.6 Satellite Communications Satellites communications rely on objects orbiting the earth to receive, amplify, and transmit signals between points that may be BLoS, as illustrated in Figure 13. Satellite communication, unlike cellular networks, provides global coverage and does not require fixed, terrestrial infrastructure. Satellite communications can provide data rates of up to 1 Mbits/s, which is suitable for PML [70]. Satellite applications have, in the main, been in navigation (i.e., GNSS) and television broadcasting. The operational requirements and miniaturisation of MAV limits their use of satellite communications. Initially, in urban environments, signal masking and multi-pathing effects are likely to be experienced below rooftop level [71]. Satellite links are not available in indoor environments. Powerlines are expected to produce electromagnetic interference which may create unreliable signals. The SWaP limitations of MAV restrict the size and numbers of antennas that can be used. Satellite communications are inherently delayed due to the long distances travelled by the signal. This might be suitable for video or sensor data transmission, but it is problematic for safety-critical CCL and PML. The data rates of satellite communication also degrade for mobile terminals (potentially down to 256 kbits/s) [70]. 19
Existing satellite-frequency allocation is inconsistent with the ICAO-defined protected aviation spectrum [59]. The technical requirements of using satellites for BLoS CCL were not defined and understood until recently. Existing technical studies [54] [62, 72] are limited to medium and large UAS operating at altitudes above 3,000 ft. These studies do not address the operational and miniaturisation requirements of MAVs. It is clear that MAV-specific technical analyses (e.g., antenna size, positioning, altitude, and short-term and long-term interference) and flight tests of scenarios are required to assess the suitability of satellite communications. Hybrid-communications protocols that combine different technologies are recommended for consideration to overcome their limitations. Integration of satellite and cellular networks was proposed by Iapichino et al. [73]; and combining satellite networks with terrestrial direct links was proposed by Kerczewski et al. [72]. 2.3
Energy Storage and Power Generation
(W/kg)
The endurance of MAVs is primarily limited by the available on-board energy. Limited energy storage poses restrictions on the amount of data that can be relayed to the GCS or other agents, as discussed in the previous section. Most MAVs employ electrical propulsion systems, despite the poor specific energy of batteries. This is due to the ability to miniaturise electric propulsion for MAV applications with minimal loss of efficiency. Additional factors are the relative ease of electric energy implementation, the fast discharge rates of lithium polymer (LiPo) batteries, and the ability to recharge batteries. Fuels such as methanol and gasoline can provide higher energy densities, but the size, weight, and complexity of the associated propulsion systems are not ideal for most types of MAVs. The slow response of fuelbased propulsion systems is also unsuitable for multi-rotor MAVs, which rely on the fast response of electric motors for attitude stability; although hybrid systems may be beneficial (citation needed??). Furthermore, fossil-fuel-based propulsion systems generally cannot benefit from airborne recharging methods, which are discussed later in this section. The use of LiPo batteries has grown rapidly [74]. Compared with alternative, commercially available technologies, they combine cost effectiveness with acceptable specific energy (approaching 200 W∙h/kg), as shown in Figure 18. Batteries generally constitute up to 40% [75] of a MAV’s mass (depending on its type and configuration) to achieve nominal flight times of 15–60 min for rotary- and fixed-wing types, respectively [31, 76]. LiPo batteries are characterised by discharge rates suitable for peak-power demands of electric motors. In contrast, fuel cells have lower discharge rates, but significantly higher energy densities.
Figure 18. Battery specific energy, power and discharge rates, reprinted from [77]
For electrically powered MAVs, based on the sizing equations provided by Brandt [78], it is possible to determine the ratio of battery weight to aircraft weight. Battery-to-aircraft weight =
V Tflight 𝑔 𝐿 ηmotor ηprop 𝐷 e
,
(7)
where V is the mean airspeed (m/s), Tflight (h) is the endurance, ηmotor is the motor efficiency, ηprop propeller efficiency, g 20
(m/s2) is the acceleration due to gravity, L/D is the lift-to-drag ratio of the aircraft, and e (Wh/kg) is the specific energy of the battery. Figure 19 shows the relationship between endurance and required specific energy of a battery. This is repeated for different aircraft designs indicated by L/D ratio. This formula assumes that the payload is not consuming power from the energy source. Figure 19 shows estimates of the endurance of an electrically powered MAV based on its L/D and the specific energy of the on-board energy storage, computed by use of equation (7). Two examples of energy storage are shown: fuel cell (high specific energy) and LiPo battery (low specific energy). Based on the estimated L/D (highlighted in green) for multi-rotor MAVs, their endurance using LiPo can be estimated to be up to 0.5 h compared to 1.5 h for fuel cells. This graph also serves to illustrate the influence of specific energy on endurance due to the SWaP limitations of MAVs. A conservative estimate of efficiency for a brushless DC motor (0.7 [79]) was used; propeller efficiency was taken as 0.8 [80]; and MAV’s airspeed was assumed to be 10 m/s [81]. The battery-to-aircraft weight was maintained at 40% [82].This estimate caters for avionics, airframe and, crucially, payloads. It is important to note that fixed-wing MAVs can attain a range of L/D ratios (1–10) depending on their design [83, 84]. A reference L/D ratio associated with highefficiency manned sailplanes (L/D = 30) is also included in Figure 19, as a baseline for maximum typical aerodynamic efficiency.
Figure 19. Specific energy (Wh/kg) needed for different aircraft (L/D), as a function of endurance, for MAVs in which batteries constitute 40% of the aircraft weight
Energy density has indirect effects on autonomy, as it limits the flight time, utility, processing, sensing, and decisionmaking capabilities of the aircraft. Current research in battery technology promises to drastically improve energy efficiency [85-87], whilst battery prices are expected to continue to drop [88], as illustrated by the projected values in Figure 20. The use of graphene-based composites for Li-ion batteries has been shown to create exceptional energy densities [89, 90]. In addition, such batteries can be shaped to serve as structural load-bearing components [91].
21
Figure 20. Projected Li-ion battery costs [88] and micro-battery energy density [92] as functions of time
2.3.1 Unconventional Energy Transfer Unconventional approaches focused on energy harvesting from the environment such as thermal soaring [93-100] and orographic-lift soaring in urban environments [101-103] rely on the presence of winds and certain weather phenomena. MAVs with windmilling propellers capable of regenerative soaring have been proposed [104], as has harvesting solar energy [105]: however, the small surface area available for photovoltaic cells (i.e., solar panels) on a MAV make this challenging. Nonetheless, small solar panels may be exploited for wireless power transmission by aiming a highpowered laser beam from the GCS towards a receiver on the aircraft [106, 107]. This emerging technology has shown the potential to significantly extend endurance and to be capable of supplying a MAV with the required power [108110]. The challenge associated with this recharging technique is the high tracking accuracy required for the beam to reach the designated receiver and to avoid accidentally destroying the MAV. A similar concept of tethering a UAS to its GCS to provide, in essence, unlimited endurance has also been proposed [111]. Both tethering and laser beaming require the MAV to maintain LoS and to be within close-range of the GCS during charging; however, the MAV can fly away from the beam (or disconnect the tether) when needed. MAVs can also be charged from airborne platforms, a concept recently demonstrated by Wilson et al. [112]. Battery dumping was proposed as a means of reducing the aircraft weight once a battery is discharged [113]. Figure 21 summarises the different recharging and energyharvesting techniques for MAVs.
Figure 21: Unconventional UAS recharging and energy harvesting
2.3.2 Power and Energy Management Researchers have also explored different techniques to reduce and manage on-board power consumption. An 22
example is to explore novel aircraft designs, which improve aerodynamic and mechanical efficiencies, thus reducing power consumption. For instance, ducted omnicopters [114], bio-inspired flappers [115], and titling multi-rotor [116] concepts have been proposed. Further gains can be achieved through smart power management of multiple energy resources. Bohwa et al. [117] demonstrated the advantages of utilising an active power management system for optimising the combination of solar cells, fuel cells, and batteries on-board UAS. Accurate models of electrical load and demand profiles are essential to this approach. For instance, fuel cells have high energy densities, but low discharge rates (suitable for powering payloads). In comparison, LiPo batteries have marginally acceptable energy densities, but high discharge rates (suitable for powering motors). Therefore, an active power system is critical to managing the different power sources based on their characteristics and mission power demands [118-120]. Multiple power and management systems add complexity and hardware to the MAV. Charging, conditioning, protection, switching, and regulation components are required to support more complex power systems, limiting their practical use on-board MAVs. Efficient designs, energy harvesting, and improvements in battery technology will undoubtedly contribute to enhancing the autonomy of MAVs. Increasing efficiency will extend flying time and consequently improve the utility of MAVs in complex and remote environments. Additionally, enhanced energy storage will allow for more computations on-board the aircraft and will increase both the quantity and quality of information relayed back to the GCS. Ultimately, as battery technology improves (and energy density increases), MAVs will have battery-to-aircraft weight ratios that will enable longer endurance, thus improving the utility and complexity of MAV operations. This will allow the payload-to-aircraft weight ratio to increase, providing for a greater variety of sensors, payloads, and sophisticated flight-control systems to be carried on-board the aircraft. It is the payload that determines the utility of a MAV; therefore, the ability to carry a wide range of payloads is essential for an autonomous platform.
3
Sensing
MAVs, like all robots, use various sensors (1) to perceive the environment (as inputs to path planning, mapping, etc.) and (2) to estimate the pose (position and attitude). Figure 22 provides a framework for classifying sensors used in pose estimation, as proposed in [9, 121] and summarised in Section 3.1. Sensors are categorised into environmental awareness, pose estimation and sensors that can be used for both purposes (common sensors). However, a robust framework to characterise sensors used for environmental awareness has not been developed. The most relevant work was conducted by Chen et al. [122] for robotic vision. Therefore, Section 3.2 creates a framework for evaluating environmental awareness sensors and provides a qualitative evaluation.
Environmental Awareness
Common Sensors
Acoustic Sensors
Optical Sensors
Sonar
LiDAR
InfraRed
Mono / Stereo Vision
Pose Estimation Translational / Rotation Sensors
Optical Flow
Magnetometer (compass)
GNSS
Inertial Sensors
Accelero meters
Gyroscopes
Flow Sensors
Pressure Sensors
Figure 22. Sensor categorisation for autonomous MAVs
3.1
Pose-Estimation Sensors
The operational environment of MAVs is turbulent, especially in close proximity to buildings and other large structures. Whilst wind engineering has been studied for the last fifty years, most work has concentrated on: wind loads on structures [123, 124]; the dispersion of pollutants downwind of structures [125-127]; pedestrian wind comfort [128, 129]; and the effect of wind on the operation of airport runways [130, 131]. Most studies have been conducted in wind tunnels or through computational simulation, though rarely with full-scale flow measurements [132]. Attempts are now being made to use MAVs to measure and map the flow fields around buildings [133]. Turbulence intensity rapidly increases with decreasing altitude [134 272]. Current MAVs therefore face operational constraints due to severe turbulence at low altitudes. Hence, increasing the operational spectrum of MAVs in turbulent conditions is essential to enable them to safely conduct autonomous missions in urban environments. The large turbulence scales relevant to the miniature size of the vehicle, in addition to the fast perturbation rates, can degrade the pose estimate significantly, thus degrading flight-path and attitude-tracking performance. This degradation can be hazardous for low-level flight in complex environments and is particularly significant for under-actuated vehicles, which can undergo significant flight-path deviation before returning to a planned path.
23
Figure 23: Turbulence-intensity profiles for a range of terrain roughness. Adapted from [134].
Pose estimation is of increasing concern with the introduction of progressively smaller UAS and is considered critical for MAVs, which face significant stability issues in turbulent flow environments [121, 135-140]. Pose estimation involves localisation and attitude sensors, which, respectively, can be used to determine the position and orientation of the vehicle. Inertial navigation units (INUs), which employ a combination of inertial, rotational, and translational sensors, are commonly used for pose estimation. Inertial-based sensors can often lack the update rate, resolution, accuracy, precision, robustness, and reliability needed for MAVs [121]. Sensor fusion, in combination with efficient and effective algorithms, can lead to substantial pose-estimate enhancement in addition to bounding accumulated errors. Attempts to also develop automated tuning algorithms that can be implemented on-board the vehicle may provide further enhancements [141]. The utility of optical sensors for pose estimation is critical for automatic guidance of aerial vehicles that require decision-making autonomy [142, 143, [144]. Through a bio-inspired image-processing approach for visual guidance known as optical flow, translation and rotation in a series of consecutive images can be detected. Optical flow can be described as the apparent visual motion in a scene as seen by a moving observer. It is perceived by the eye as a vector field representing angular speed, at which a contrasting object in the scene is moving past the observer [143]. Advancements in the performance of charge-coupled device cameras and digital-signal processors have aided progress in the implementation of optical flow [145]. This has facilitated the utilisation of optical flow in small UAVs on which an embedded computer processing chip is dedicated entirely to the determination of displacement measurements [146]. There have also been recent attempts to utilise multiple sensors for performance improvement of optical flow [147, 148]. A major limitation to optical flow and other electro-optical (EO) approaches is that their performance is highly dependent on the visual scene. Lighting conditions (e.g., glare or low light), lack of discernible or distinct features, and weather conditions (e.g., rain or fog) can degrade their performance. This limits a MAV’s utility in certain conditions and reduces its operational spectrum. Despite major advances in computer-processing hardware, optic-flow approaches are computationally intensive; and this may introduce undesirable latencies. The additional computational hardware required to process optic-flow measurements means that they have higher relative power consumption than many other sensing options. Optical flow may also be constrained by the bandwidth of the input visual data [149]. MAVs have been reported to perturb at rates of up to 60 Hz in the presence of severe turbulence [81]. This implies that the video-sampling rate required is at least 120 Hz, accounting for aliasing, because the update rate of the computed pose estimate must at least match the perturbation rate. Computational systems meeting the SWaP limitations of MAVs and capable of processing 120-Hz video feed and outputting a pose at 60 Hz do not yet exist. Further research addressing practical problems associated with MAV implementations of optic flow is therefore needed. Another emerging area of research involves fusing the output of flow sensors to the INU to enhance pose estimation [150]. Flow sensors capable of sensing flow disturbances and control systems capable of counteracting them before they induce a perturbation are desired. The concept has been termed phase-advanced sensing due to its potential to account for latencies of the control system in producing a counteracting response. Such systems have been shown to improve pose estimation and thus to enhance attitude stability [133, 151-154]. The minimal processing required to produce an output makes this sensory technique ideal for enhancing the pose estimates of INUs. This emerging area of research may provide the sensory needs for MAVs to fly in severe turbulence. 3.2
Environmental Awareness Sensors
This section describes a variety of environmental awareness sensors, based on the limitations outlined in Section 2. 24
Consequently, a qualitative assessment based on the criteria outlined below is used. A quantitative evaluation was avoided because the performance of sensors is varied, i.e., is vendor-specific and dependent on the current technological state, which is expected to improve over time. The adopted assessment criteria (Table 1), is based on that proposed by Mohamed et al. [9]. The assessment is made on an ordinal measurement scale, which has the qualitative values of Unacceptable, Marginal, and Acceptable. The complete sensor evaluation matrix is presented in Table 2; and the following subsections describe the evaluation of each sensor type in more detail. Table 1: Evaluation criteria for environmental awareness sensors
Criteria
Description
Look-ahead distance Latency Coverage/availability Robustness Size/weight Computation Power consumption Resolution Sensitivity Accuracy/precision Drift S/N FoV
The difference between the maximum and minimum distances of the sensor measurement. This can be considered as the distance-measurement range A measure of delays incurred because of sensor measurement processing The ability of the MAV to use the sensor in different environments and conditions Resistance to impact and harsh outdoor conditions Sensor suitability relative to the limited payload storage space and mass of MAVs Processing required to produce a measurement output Electrical energy required to operate the sensor and process its measurements The degree to which an input can be detected The smallest change in input that can be detected The amount of uncertainty and reproducibility associated with measurements A sensor’s gradual degradation from the initial calibrated state A measure of the level of the desired signal to the level of background noise The angular range that can be detected
FoV
S/N
Drift
Accuracy/precision
Resolution/sensitivity
Power consumption
Computational load
Size weight
Robustness
Coverage/availability
Environmental sensors
Latency
Look-ahead distance
Table 2: Evaluation matrix of environmental awareness sensors
Acoustic sensors Sonar Optical sensors IR range Thermal imaging
HD
Monocular vision
HD
Stereo vision
HD
LIDAR Unacceptable Marginal Acceptable Hardware dependent
HD Marginal
Significant Challenge
Acceptable
Performance Evalutation
Best Imaginable
Worst Imaginable Descriptive Ratings 0
10
20
30
40
50
60
70
80
Figure 24. Evaluation of MAV-compatible sensors
3.2.1
Acoustic Sensors 25
90
100
3.2.1.1 Sonar ranging sensor Sonar sensors rely on emitting pressure waves (referred to as probing pulses) and measuring the time taken for the wave to return to a receiver. This concept is often referred to as time-of-flight (ToF) sensing. The frequency of the transmitted wave is upward of 40 kHz, hence the name ultrasonic sensors. There are two types of sonar-transducer technologies: piezoelectric and electrostatic [155]. Electrostatic sonars function by vibrating two voltage-biased plates and changing the capacitance between them. Piezoelectric materials can also be used, as they vibrate when voltage is applied across the material and vice versa. The operational bands of piezo-based sonars are limited to the piezoelectric material’s oscillation frequency. MEMS fabrication improves the performance of electrostatic sonars by increasing their bandwidth, through micromachining of a thin nitride membrane [156]. The operational frequencies of MEMS transducers can therefore be tailored to match the acoustic impedance of air to achieve higher performance. Sonars are suited for MAV operations, because they are resilient to routine operational impacts and relatively affordable. Their power and range are directly proportional and entirely hardware-dependent. However, a sonar sensor suitable for MAV operations is generally limited to ~6 m [157]. They are not sensitive to the colour or reflectivity of the detected object. Nonetheless, they are generally suited for sensing obstacles normal to the direction of pulse propagation. Angled, smooth objects may lead to redirection of the beam, resulting in false readings of an incident object. The reflected signal can also be particularly sensitive to the target area, relative object attitude, and the ability of the material to absorb sound energy (such as foam). The reliance on acoustics is particularly limited in sensing. Ultrasonic sensors are generally immune to background audible noise. This is not the case for high-frequency noise that may be induced, for instance, from propeller motion. Thus, sensor placement may also present an issue. Environmental changes such as pressure, altitude, and temperature variations alter the speed of sound and consequently the readings. Additionally, operating in turbulent or particle-dense (scattering) atmospheres will also influence the sonar readings. Sonars have high latency compared with other range sensors. The first cause is that the transducers must be stopped prior to receiving a signal to avoid interference, which is known as the blanking time. This is in addition to the reliance on sound, which is physically limited compared to electromagnetic waves. The combination of limited range and high latency of sonars may render them unreliable for the safe navigation of the MAV. These sensors are often used for obstacle detection where the distance of a measured obstacle can be inferred from the ToF measurement. Mapping can be also be achieved by utilising a sensor array or a single rotating sensor to acquire a 360° scan. The scan can be then used to build a map of the surrounding obstacles [158-160]. Scan patterns could be utilised for detection of features such as corners and particular geometries. More recently, Steckel and Peremans [161] proposed a novel dual sonar receiver arrangement for indoor simultaneous localisation and mapping (SLAM). The authors did not identify research that relying on sonars for field or outdoor usage. This is attributed to the limited range, high latency, and noise sensitivity exhibited by those sensors. 3.2.2
Optical Sensors
3.2.2.1 Infrared-range sensors Infrared (IR) range or proximity sensors are commonly found in mobile robots and in consumer electronics. They may be constructed using pairs of light emitting diodes (LEDs) and phototransitors, diodes that act as sensors for the light emitted by an LED. The LED is programmed to emit an IR signal at a certain wavelength (typically in the near-IR band, at ~710 nm), and the phototransistor sends an electric signal when it is exposed to light of that frequency. The operation of IR proximity sensors is conceptually similar to the time-of-flight sensing used by sonar and light detection and ranging (LIDAR) sensors that utilise different media. The advantages of IR range sensors are their low SWaP, cost effectiveness, low latency, and robustness. On the other hand, they suffer from a detection range limited to 1–5 m (for high-powered sensor). Their FoV is typically limited to a few degrees. As the detection distance grows, the signal-to-noise ratio (S/N) decreases and the readings become less reliable. Their outdoor use is limited to low light conditions due to the interference with IR radiation emitted by the sun [162]. This in turn, limits the use of increasingly popular RGB-depth imaging sensors, which have been used for indoormapping and navigation applications [163]. They are also highly sensitive to the reflectivity and direction of motion of detected objects. IR proximity sensors are more oriented towards obstacle detection. Sensor arrays could possibly be arranged around a MAV structure due their beneficial SWaP characteristics. They are often used for obstacle avoidance and tasks such as GNSS-denied altitude estimation. For MAVs they are better suited for tasks in which visibility is limited such as in smoke-filled buildings [164] and low-lighting/night operation.
26
Figure 25. Some IR range sensors are limited to indoor use due to interference with electromagnetic energy emitted from the sun
3.2.2.2 Thermal imaging Thermal imaging cameras detect objects radiating heat (electromagnetic energy in the long-wavelength infrared at ~9– 14 µm) from their surroundings. They consist of a lens, an IR-sensitive array, and a signal-processing unit. Often a cooling unit is required to reduce noise in the resulting images. They offer more robust sensing in varying conditions than do night-vision (i.e., low-light) or traditional EO sensors because they do not rely on visible light. All cameras require a minimum amount of light and contrast to enable detection of different objects. Nonetheless, EO sensors remain prevalent in robotics applications. Thermal imagers have been traditionally utilised as payloads for the human operator of an RPA to use in addition to imaging and mapping applications. Currently researchers are investigating the use of thermal imaging for fire detection and pattern recognition for SLAM [165]; however, the size of these sensor limits their use on MAVs. Despite the miniature size of thermal-radiation sensors, the size of the optics (lenses) is significantly larger. The focal length, which is the distance between the lens and the sensor, is larger than that of typical EO cameras (200–500 mm, compared to ~50 mm) [166]. Consequently, the FoV is more restricted and the lens diameter must be increased proportionately, to maintain the quality of the image. For longer-range sensing or highersensitivity applications, a cooling unit may also be required, which would increase the weight, size and power consumption of the unit. 3.2.2.3 LIDAR LIDAR sensors used for range measurements typically operate in a similar manner to IR-range and sonar sensors, i.e., as ToF sensors. However, as illustrated in Figure 26, they rely on a coherent beam. Therefore, unlike IR proximity there is limited divergence from the source. This enables ToF sensing using the phase shift between the emitted and returned signals. LIDAR was initially developed for atmospheric sensing and topographic mapping and has been propelled by concurrent advancements in GNSS and INU. Sabatini et al. [167] conducted a detailed review of the use of laser-based sensors in aircraft systems. For two-dimensional range scans, a large number of pulses are emitted per unit time (30–1,000 pulses/s); and the reflected waves are captured by the sensor. In many cases, one or more sources are combined by use of rotating mirrors to perform each scan, as illustrated in Figure 27. LIDAR instruments have been introduced for threedimensional scanning with multiple sources, angled rotating mirrors, and rotating housings. The principal mechanism of operation for LIDAR sensing results in large data sets and a commensurately high processing load. The intensity of the returned wave is dependent on the reflectivity of the detected object and may be affected by ambient light sources (e.g., sunlight). Nonetheless, LIDARs are less sensitive than EO sensors to environmental factors and can also relay depth information. EO sensors rely on passively sensing ambient light. In contrast, LIDAR is an active sensing mode. LIDAR ranging consumes more power, particularly as the measurement range and scanning frequency increase. Another aspect that must be considered with the use of lasers is safety, as MAV will operate in urban environments. Regulations govern the various classes of lasers to which humans may be exposed [168]; and eye-safe LIDAR systems have been developed..
27
Figure 26. Coherent and non-coherent beams
Traditionally LIDAR scanners were large, heavy, and power intensive; however, miniature LIDARs particularly suited for MAVs operations in terms of SWaP have been developed [169, 170]. The range and mass of several such LIDAR instruments are illustrated in Figure 28. Miniature LIDARs rely on rotating mirrors and a slotted, fixed head, as opposed to the traditional rotating head design illustrated in Figure 27. Currently, these products sacrifice other operational requirements such as cost, robustness, accuracy, sensitivity (outdoor use is limited), and range (less than 50 m). Outdoor use of miniature LIDARs is limited by solar interference. It is expected that advances in miniature LIDAR will address these issues.
Figure 27. Construction of a LIDAR scanner
35 Hokuyo LIDARs
30
UTM
Sick LIDARs
Range (m)
25 UST
20 15 TIM3/5
10 URG TIM3/5
5
UBG
0 0
100
200 Mass (g)
28
300
400
Figure 28. Range and weight of various 2D LIDAR scanners suited for MAV applications [169, 170]
3.2.2.4 Cameras Images captured by one or more on-board EO cameras can be used for both environmental awareness and pose estimation [9, 121]; and computer vision has become an essential aspect of robotics and automation. These techniques are used for feature extraction, target detection and tracking, and mapping [149]. Motion estimation is performed by comparing sequential frames to measure the displacement between detected objects. SLAM algorithms relying on feature extraction for landmark detection are referred to as visual SLAM techniques. Current research is focused on improving practical implementation of theoretical developments, particularly dealing with uncertainty and cooperative mapping [122]. Computer vision is a broad and evolving field that has been discussed in surveys on robotics [122, 171]. Particularly for MAVs, the use of vision for perception is appropriate, as visual sensors are compact in size and consume less power than do active ranging sensors. Nonetheless, there are associated challenges that must be addressed with vision-only solutions. First, the computational load of image processing and map building is demanding for limited on-board resources. As with any computationally intensive task, this is often handled by relying on powerful off-board computers, which leads to data latency and increased power consumption (Section 2). Additionally, for visual data, the quality of video transmission is governed by the Johnson Criteria, which set resolution thresholds for object detection, orientation, recognition, and identification [172]. Compared with LIDAR scanners, cameras have a limited FoV; and this limits the reliance on vision-only solutions, particularly in dynamic urban environments. Prototypes that combine multiple EO sensors and curved lenses to enhance the FoV have been demonstrated and will likely play an important role in the utility of vision-based systems [173]. Vision-based sensing can suffer from sensitivity to external changes in environmental conditions and disturbances. A single camera is not suitable for MAV environmental-awareness applications, such as mapping, because instantaneous depth information is not captured. A complementary sensor is therefore needed. Researchers have developed solutions using two cameras (stereo vision) [35] and monocular cameras with INUs and/or range sensors [36, 174, 175]. Kinect and other RGB-D sensors are increasingly popular in robotics because they provide image (single camera) and depth information (via an IR projector and receiver); however, they are limited in range and their use is compromised by strong sunlight [162]. The use of a single camera with an INU is more practical for MAVs because most on-board flight-control systems are equipped with INUs. Consequently, this leads to a reduction in payload mass and energy consumption. On the other hand, INUs are typically ill suited for MAVs operations [81, 121], because of their relatively high levels of uncertainty. 3.2.3 Future Trends It is evident from the above evaluation that there is no singular sensing solution ideally suited for MAVs. Several researches have presented solutions that rely on a combination of multiple sensors to provide sufficient information for safe navigation and path planning [27, 32, 35, 36, 162, 176]. There is no doubt that sensor fusion and filtering techniques are required for MAV perception. The concept of sensor fusion is illustrated in Figure 29. The issue that arises is the selection of the appropriate sensors for a particular mission (given duration, distance, and environment requirements) and aircraft characteristics (payload, on-board computer, and on-board energy limitations). The sensor selection process must consider all their inherent limitations, as shown throughout this study. Among the most relevant considerations is the ARR, introduced in Section 2.1.3. Based on the the look-ahead distance of the sensors (Figure 30), the ARR was computed as illustrated in Figure 11. The presented ARRs are idealised, through the assumption of static obstacles, and can be useful for fixed-wing MAVs, which travel at a forward speed and are typically unable to hover. In a real environment, dynamic obstacles can appear in the path of a MAV at any distance. Rotary-wing and recently introduced hybrid fixed-wing MAVs (which can transition from level flight to hovering) are able to “pause” their flight through hovering, while a path is being computed around a dynamic obstacle. Current sensing solutions are not suited for autonmous operation of MAVs in urban environments. Further research on sensing technology is require to accommodate the previously addressed unique constraints of MAVs. Novel solutions are needed to reduce the power consumption, computational complexity, bandwidth limitations, and minituarisation requirements that restrict MAV operations. Promising solutions have been proposed. For example, light-emitting dode detection and ranging (LEDDAR) [177, 178] technolgy utilises separate LED beams to reduce computational complexity and power consumption and to increase the range of time-of-flight measurements. LEDDAR sensors are limited by beam size, which reduces the sensor resolution at long distances. Chip-based radar technology [179-181] integrates a radar system on a single complementary metal–oxide–semiconductor (CMOS) chip. It has been shown to reduce power consumption and production costs, whilst maintaing a target-detection range of several hundered meters [181]. Radar performance is currently limited for slow moving targets. Single-photon avalance diodes (SPAD) are capable of combining images with range data for individual photons, essentially generating 3D images [182]. Recently, mulit-channel solid-state SPAD sensors have been developed to improve image and timing resolution [183]; however a compromise between spatial and temporal resolution is required [184]. The aforementioned technologies are still under development; and thus application-specific evaluation and technology demonstration are critical for autonomous MAV applications. 29
Figure 29. Environmental-awareness sensor fusion
30
Figure 30. Time-of-flight measurements are typically conducted using ranging sensors such as sonar (acoustic), LIDAR (laser-based) and optical (IR) sensors
4
Planning
Planning can be defined as the process of generating a set of continuous, executable actions from the current state (configuration/location/orientation) towards a desired state (configuration/location/orientation). It is an essential task for any mobile robot, flying or otherwise, and has been widely studied in robotics. Early mobile robots and planning algorithms were purely reactive because actuation relied on limited sensing [185, 186]. For an autonomous platform, at least two hierarchal levels of planning are needed; namely higher-level behavioural (or mission) planning and vehiclelevel (or motion) planning, as illustrated in Figure 5. The motion planning aspect can be further divided into deliberative and reactive levels. Attempts to utilise generic robotic approaches to MAVs have been unsuccessful, as a consequence of their particular behaviour and the nature of their operational environment. Steering methods and simplified models have been proposed for autonomous cars and robotic arms to facilitate their automation. The sensitivity of MAVs to external, highbandwidth perturbations further complicates their modelling, planning, and control [9, 81, 121, 133]. Simplified vehicle characterisation and environmental sensing (e.g., obstacle detection and target tracking) may be feasible for larger ground robots functioning in ordered environments. For a MAV operating in a cluttered and turbulent environment, such simplifications may lead to mission failure by designing colliding or non-feasible flight paths. Adapting the planned paths to the dynamic environment and the non-linear aircraft aerodynamics associated with the flight regime of MAVs is essential to ensure safe navigation. As illustrated in the previous sections, limited payload SWaP poses additional constraints on MAV autonomy as limited sensors, batteries, and flight computers can be carried on-board. Environmentally induced perturbations [121, 187] can lead to variations in MAV flight path, which cannot be accurately predicted and accounted for in an off-line planning stage. Thus, off-line plan execution becomes an unnecessary task, as active planning and actuation are still needed. The combination of a highly dynamic environment with a sensitive vehicle and limited computational resources presents a challenge for the autonomous planning systems. This paper is focussed on software developments and methods that are particular to MAVs, thus, this section describes high-level behavioural planning and low-level motion planning. MAV-specific challenges are discussed, including only algorithms that address these challenges or algorithms that have been flight tested with MAVs. This section is dedicated to software developments in high-level, path, or swarm planning for MAVs. There has been limited development in decision making and machine learning that is particular for MAVs (Section 4.1). Advancements in algorithms for target tracking, image recognition, and image processing are reviewed in [122]. These algorithms are not discussed here, as the advancements are not specific to MAVs and are in development for mobile robots. 4.1
High-Level Behavioural Planning
The autonomy level of a MAV is subject to its decision-making capabilities with respect to desired mission goals [16]. This is often referred to as contextual intelligence. MAV applications are rapidly increasing and expanding into unique applications targeting a number of industries including agriculture; cinematography; law enforcement; buildings and construction, environmental conservation, and weather monitoring. From a planning perspective, the agent’s behaviour can be categorised into exploring unknown environments [188-190], target tracking [144, 191-193] and waypoint navigation [194-196]. A high-level planner is required to assess the current situation and strategically redefine a desired location or behaviour. This class planner is often referred to as a mission planner, high-level planner or behavioural planner. Nonetheless, they all perform multi-objective decision making in stochastic environments and with incomplete knowledge of the situation. To perform at this level of strategic decision making, the planner must assess: 1. mission-critical data, such as structural health, motor failure, and energy state 2. civil-aviation rules and regulations (Section 1.1.2 of this paper) 3. time-varying environmental features, such as on-coming or neighbouring aircraft, wind gusts, and reduced visibility 4. cost/reward of each plan/policy, which may include time, energy consumption, risk, uncertainty, distance, wind condition and rain. This problem could be modelled as a Markov-decision process in which the planner attempts to define a policy that maximises a reward (or an objective function) in a stochastic environment, e.g., [13, 197, 198]. Fuzzy-logic control has also been used for behavioural planning because of its ability to mimic reasoning by formulating sets of rules with linguistic representations [199, 200]. Finite-state machines have been developed to nominate different states that define the agent’s behaviour [201-203]. For instance, a state could be terrain-following behaviour, whereas another could be loitering. The decision-making process is often modelled as a multi-variate, multi-objective optimisation problem and solved using evolutionary algorithms [204, 205]. Other optimisation and decision-making methods, such as ant-colony optimisation [206], simulated annealing [207], and machine learning [208], are yet to be utilised for MAV decision making, but have been beneficial in other fields. Behavioural planners are solely responsible for cognitive decision making and are not concerned with the actuation or execution of those desired goals or tasks. 31
There is a large body of work on the high-level task planning for air traffic management (ATM) applications [209]. However, this topic remains a research frontier with limited research on MAVs in urban operations. The majority of planning research for MAVs is limited to simplified, controlled environments (i.e., 3D or 2D) without replanning as discussed in the following section. 4.2
Motion Planning
Motion planning is necessary for enabling high LoA, by achieving RP independence and carrying out missions in complex environments, as defined in Section 1. It is also a central task for any autonomous system, as illustrated in Figure 5. Planning is a widely studied topic in robotics, particularly for industrial manipulators, computer animations, and mobile ground robots. The simplest form of path planning is purely geometric: the environment is assumed to be static and known, and the robot is assumed to be a rigid structure. Early efforts to develop deterministic solutions, even for the simplest cases of geometric planning, were found to be computationally exhaustive [210]. 4.3
Challenges of MAV Planning
MAV planning is a particularly challenging task because it combines the high dimensionality of aircraft with a highly dynamic and cluttered environment. Additionally, MAVs are particularly sensitive to flow disturbances, which introduce further complications. Therefore the planner’s search is conducted in a highly dimensional, restricted, and time-varying state space. The challenges are summarised below: • Highly dimensional state space: The aircraft’s configuration could be described using a combination of position (x, y, z) and attitude (roll, pitch, yaw) with respect to a reference global frame. To consider velocities in planning, a 12dimensional state space is attained. • Under-actuation: MAVs are under-actuated because they are typically limited to four input commands to control an aircraft with six degrees of freedom. A fixed-wing MAV has four inputs, namely throttle, aileron, rudder, and elevator, and is consequently restricted in its ability to explore the full state space. On the other hand, the unique agility of multi-rotors can be classified as ‘trivial under-actuation’, because the motor thrust (input) allows it to easily explore the state space. • Nonholonomic constraints: These refer to non-integrable velocity constraints that arise from the kinematics of the aircraft. For instance, the limited turning radius of a fixed-wing aircraft restricts its ability to navigate to a desired location and restricts exploration of the state space. • Dynamic environments: MAVs are designed to operate in urban cluttered environment at low altitudes. This is far more challenging than larger UAS or ground vehicles. Large UAS operate in open and stable environments, whilst ground passenger vehicles operate in structured areas. The inherently small design and low inertia of MAVs increases their sensitivity to disturbances, which requires the planner to be constantly regenerating plans to accommodate the unpredictable changes in the environment.
32
Figure 31. Motion planning to accommodate MAV operations, particularly in obstacle-rich urban environments with dynamic and often unpredictable occupants
4.4
MAV Specific Planning
This section describes the need for MAV-specific planners, based on their unique constraints and the corresponding limitations of current state-of-the-art planning methods. 4.4.1 State-of-the-Art Motion Planners Several planning classifications have been proposed. An initial distinction is that planners either generate paths, with no time component defined, or they generate trajectories with an explicit execution time. The most common classification is the manner in which the search is conducted. Latombe [211] introduced this classification, and it has been widely adopted in the literature. There are other classifications (such as deterministic, probabilistic, and search space). Accordingly, planners could be categorised into roadmap, potential field, graph search, optimisation, and randomised-search algorithms (Figure 32). Roadmap methods attempt to capture the connectivity of the robot’s free environment into graph structures such as Voronoi diagrams, cell decomposition, and visibility graphs [212-215]. A* and Dijkstra graph-search algorithms are predominantly used after the environment has been converted into a graph structure by a roadmap algorithm. These methods are mostly suited for static and limited search regions, as computational complexity increases exponentially with the dimensions of the search space [33]. More efficient and robust heuristic-based incremental graph search algorithms were recently proposed. These include Dynamic A* (D*), Lifelong Planning A* (LPA*) and Anytime Dynamic A* (AD*) methods [216-218]. Potential-field methods utilise artificial forces towards the goal location to attract the robot and local repulsive forces away from the obstacles are utilised to prevent collision [219]. These methods suffer from two major drawbacks: oscillating paths in narrow regions and failure caused by local minima [220]. In the last decade, randomised-search algorithms have been developed to overcome the high dimensionality of planning [221]. Most widely used are Probabilistic Roadmap Method (PRM) [222] and Rapidlyexploring Random Tree (RRT) algorithms [223, 224], which rely on random sampling of the environment. RRT is more suited for autonomous MAVs, as it iteratively grows a path towards the goal. Randomised planners generate suboptimal paths with redundant manoeuvres, as a consequence of the stochastic sampling utilised in the search process [221]. In the next section, we provide a functional planner assessment with respect to MAV operations. It must be noted that these algorithms are not limited to 2D spaces and can be implemented for multidimensional state spaces.
33
start
end
Figure 32. Examples of path planning: (top left) exact roadmap planning using a visibility graph, (top right) discretised graph search using A* algorithm, (bottom left) potential-field planning, and (bottom right) randomised planning using a bidirectional RRT algorithm.The start and end of each path are the identical.
4.4.2 MAV Planner Assessment Most MAV planning algorithms have been tested through simulation. In this section, we present a functional categorisation to motion planning and categorise existing MAV planning methods. Planners are predominantly categorised according to the approach used to conduct the search, rather than the operation of the planner, e.g., randomised or incremental search. The criteria considered in this assessment are the environment model, aircraft model, path characteristics, and efficiency of each planner (Table 3), based on the categorisation introduced by Elbanhawi and Simic [221] for assessing sampling-based randomised planners. It must be noted that in some instances the planners shown in Table 3 were also proposed generally for small UAS and are not specifically for MAVs. The term ‘environment model’ refers to the manner through which the environmental features (e.g., obstacles and free regions) are represented by a planner. The ‘aircraft model’ represents the ability of the planner to account for the kinematics and dynamics of the vehicle. This is essential, as it enables the planner to generate realistic/realisable solutions. Some investigations have been made on the use of motion primitives to embed the vehicle’s dynamics and to lower the dimensionality of the search [225-230]. ‘Path characteristics’ refer to the quality of the generated path. This may be difficult to quantify, as the metric for the true cost of the path is often intractable [231]. A comparative study of optimal planners showed that running multiple instances of a randomised planner with smoothing provided more consistent performance in different environments [232]. Nonetheless, this evaluation was based on a predetermined metric such as path length, rather than minimum time or energy expenditure. It was also focused on unconstrained robotic designs. ‘Efficiency’ relates to the computation of the algorithm. It is essential for a planner to be capable of replanning at relatively high rates to account for the changes in the vehicle state and surroundings. Extending planning periods causes delays in the systems response, reducing the response ratio of the system and compromising its safe navigation. The resulting paths have been shown to affect the performance of autonomous vehicle control [233-235].
34
Table 3. MAV path planners Planner
Environment
Aircraft
Path
Efficiency
MAV Planner
Random search [222, 236]
Multidimensional continuous space
Point or Dynamic model
Suboptimal: Post processing for optimisation
Metric dependent
[189, 237-239]
Grid [216]/lattice search [240]
Multidimensional, discretised space
Point or dynamic model
Resolution optimal: Anytime variants have been proposed
Discretisation and dimension dependent
[37, 241]
Potential field [219]
2D/3D
Point
Suboptimal: Prone to local optima and oscillations
Environment dependent
[242, 243]
Roadmap [212214]
3D/2D discrete polyhedra
Point
Resolution optimal
Obstacle and dimension dependent
[242, 244, 245]
Inverse dynamics
2D/3D
Dynamics Model
Optimal
Optimisation algorithm and cost function dependent
[246, 247]
continuous space
continuous space
There is limited work on MAV-specific reactive planning algorithms, though robust reactive planning solutions have been proposed [248-251]. Nonetheless, they are not sufficient for autonomous robots and can only be added as a subcomponent of the decision-making process. Reliance on local planning can lead to suboptimal planning and even risk collision [252, 253]. Motion-planning algorithms that implement inverse dynamics for UAS trajectory optimisation for re-planning with dynamics obstacles have been proposed [246, 247]. There are limited numbers of MAV planners that have been trialled beyond computer simulation. Existing MAV planning implementations are limited to controlled scenarios such as fully known static indoor environments with external tracking systems or prior scene knowledge as shown by Shen et al. [190] and by Mellinger et al. [254]. The success of deploying small-scale UAS [35, 243, 255] suggests that as technology converges, autonomous MAVs will be possible. Ultimately, the close integration of a behavioural planner, motion planner, reactive obstacle avoidance, and plan execution will be essential for a successful MAV. These layers must be capable of accurately predicting MAV dynamics and sufficiently robust to maintain performance in a range of environments and situations. Considering the requirement for low altitude flying in urban environments is another challenge for MAV planning. MAVs are expected to operate in urban regions cluttered with buildings and other structures. Traditionally, UAS planning has been focused on senseand-avoid technologies, whose concern is detecting other aircraft in sparse airspace. This is significantly less challenging than the problem at hand. 4.4.3 Swarming Multi-agent swarms of MAVs pose additional challenges. A swarm of MAVs could comprise a heterogeneous mix of MAVs with a mixture of sensing, dynamics, and performance limitations (e.g., endurance limits). The goal for the swarm is to achieve a mission objective through the best use of its cooperating components. This requires a hierarchy of autonomy: autonomy at the level of the individual MAVs and emergent autonomy defined in relation to the behaviour of the swarm as a whole. From a planning perspective, a number of limitations are associated with swarming MAVs. In cluttered and turbulent environments, when one agent experiences a disturbance that results in a path deviation, this deviation could propagate throughout the swarm, depending on its interconnection architecture, as illustrated in Figure 33. Issues such as inter-agent collision detection and avoidance in turbulent conditions and deconfliction planning must be addressed. Current research in swarm planning has attempted to define conventions for confliction planning under idealised conditions [204, 256]. For instance, a proposed convention for the collision avoidance of two swarms is shown in Figure 33. These areas of research represent a significant opportunity for investigation. Standards for MAV swarm behaviour are yet to be identified and developed. Swarm planning presents a challenge for planning in terms of the scale of planning and behaviour within swarm. As mentioned previously, planning for a single MAV is a challenging task. Planning algorithms suffer from an exponential increase in planning time with increases in the dimensionality of the planning problem. Compounding the problem with multiple MAVs will further limit the usefulness of existing planning algorithms. Hence, it is not feasible to consider each MAV separately. However, replicating behaviour within a swarm might cause collisions within swams or with other aircraft, unless standard manoeuvres are defined.
35
(m)
(m)
(m)
(m)
(m)
(m)
Figure 33. Swarming conventions, reprinted from Panyakeow and Mesbahi [256]
Current research in swarming is largely limited to identical aircraft flying in formation to perform particular tasks. The identified restrictions in MAV technology indicate that such designs are not practical. Alternatively, we envision a swarm of different aircraft configurations (i.e., airframes, propulsion, and energy-storage technologies and/or capacities), with different sensors suites performing a single mission. This is denoted as hybrid swarming. Such an approach would enable the swarm to achieve a single mission with different sub-tasks, such as reconnaissance, refuelling, communication relay, and environment monitoring. Hybrid swarming would require a redesign of traditional human-machine interfaces and control stations to enable a single operator to monitor and control the swarm. Additional research effort is required to combine hybrid environment measurements into a common map. Hybrid swarm mission and path-planning algorithms require further development. Hybrid swarming is envisioned as the future of autonomous MAV operations.
5
Concluding Remarks
Fully autonomous operations represent an important milestone in the technological maturity of MAVs. This milestone is challenged firstly by the lack of consensus on the definition of autonomy, the usage of terminology, and the quantification of autonomy. To develop and evaluate commercial products and research for autonomous MAVs, a domain specific framework for measuring autonomy and a universal consensus on terminology are needed. This will also enable the development of adaptive autonomous behaviour that is capable of integrating various levels of autonomy at different stages of a mission (i.e., take-off, loitering, landing, flight-mode transition, target tracking etc.). Swarming is considered the highest level of autonomy; however, swarming behaviour remains incompletely defined. Cooperative multi-agent flocks conducting missions with limited human oversight are envisioned. Hybrid swarms combining alternative aircraft configurations and sensing solutions are foreseen as the future of autonomous MAV operations. The unique technical challenges associated with enabling autonomous MAVs in urban operation have been identified. Energy-storage capacity represents a major technical challenge. In particular, current battery energy densities restrict MAV endurance to 10–40 min. Novel energy storage and regeneration technologies could lead to significant increases in MAV endurance with the potential for MAVs to remain airborne indefinitely. Other unique technical challenges have been described in this paper, including computational power and communication bandwidth limitations, which restrict MAV sensing and planning capabilities. Existing communication technology cannot accommodate the command, control, and payload-management bandwidth and range requirements of autonomous MAVs. A combination of cellular-network and satellite technology may lead to improvements; however, research is lacking on the feasibility of this approach for MAVs operating at low altitudes in urban environments. Navigation in turbulent and cluttered urban environments with dynamic obstacles is a current limitation for autonomous MAVs. Indeed, robust obstacle avoidance is yet to be demonstrated in such environments; and this task requires improvements in sensing and planning performance. Current trends in the development of commercial flight controllers, novel sensing solutions, and communication infrastructure are expected to fulfil the operational and technological requirements for autonomous MAVs. Sensor technologies such as SPAD, chip-based radar, and LEDDAR are expected to significantly improve MAV sensing capabilities by increasing detection range and reducing computational complexity and power consumption. Randomised planning algorithms have been utilised to solve challenging planning problems and can be adopted for MAV operations planning in dynamic and urban environments. The review presented here has identified opportunities that can enable autonomous MAV operations. The UAS industry is the fastest growing sector of the aerospace industry, and autonomous MAVs form a part of that sector [257]. 36
Autonomous MAVs are expected to integrate and play critical roles within several industrial and service sectors. Increasing their autonomy is crucial to further expand their utility and to meet the rapidly emerging operational demands.
Acknowledgments This research was undertaken by the RMIT Unmanned Aircraft Systems Research Team, within the Sir Lawrence Wackett Aerospace Research Centre, at RMIT University. It was funded by; the U.S. Air Force Office of Scientific Research; the Defence Science Institute; and the Australian Postgraduate Award scheme.
References [1] [2] [3] [4] [5]
[6] [7] [8] [9]
[10] [11] [12] [13] [14] [15] [16]
[17] [18] [19] [20] [21] [22] [23]
[24] [25] [26]
T. J. Mueller, Introduction to the Design of Fixed-Wing Micro Air Vehicles: Including Three Case Studies: American Institute of Aeronautics and Astronautics, 2007. S. R. Dixon and C. D. Wickens, "Control of multiple-UAVs: A workload analysis," presented at the 12th International Symposium on Aviation Psychology, Dayton, OH, 2003. R. R. Murphy, Disaster robotics: MIT Press, 2014. Y. Cheng, P. Jiang, and Y. F. Hu, "A snake-based scheme for path planning and control with constraints by distributed visual sensors," Robotica, vol. 32, pp. 477-499, 2014. G. F. Wilson and C. A. Russell, "Performance Enhancement in an Uninhabited Air Vehicle Task Using Psychophysiologically Determined Adaptive Aiding," Human Factors: The Journal of the Human Factors and Ergonomics Society, vol. 49, pp. 1005-1018, December 1 2007. D. A. Jenkins, P. G. Ifju, M. Abdulrahim, and S. Olipra, "Assessment of controllability of micro air vehicles," in Proc. Sixteenth Int. Conf. On Unmanned Air Vehicle Systems, 2001. G. Wild, J. Murray, and G. Baxter, "Exploring Civil Drone Accidents and Incidents to Help Prevent Potential Air Disasters," Aerospace, vol. 3, 2016. L. A. Young, J. A. Yetter, and M. D. Guynn, "System Analysis Applied to Autonomy: Application to High-Altitude Long-Endurance Remotely Operated Aircraft," in AIAA Infotech@ Aerospace Conference, Arlington, VA, 2005. A. Mohamed, R. Clothier, S. Watkins, R. Sabatini, and M. Abdulrahim, "Fixed-wing MAV attitude stability in atmospheric turbulence, part 1: Suitability of conventional sensors," Progress in Aerospace Sciences, vol. 70, pp. 69-82, 10// 2014. R. Clothier and R. Walker, "Safety Risk Management of Unmanned Aircraft Systems," in Handbook of Unmanned Aerial Vehicles, K. P. Valavanis and G. J. Vachtsevanos, Eds., ed: Springer Netherlands, 2014, pp. 2229-2275. M. Elbanhawi, "Randomised parameterisation motion planning for autonomous cars," PhD Doctoral, School of Engineering, RMIT University, Melbourne, Australia, 2016. R. Clothier, B. Williams, and T. Perez, "A review of the concept of autonomy in the context of the safety regulation of civil unmanned aircraft systems," in ASSC 2013, 2013, pp. 15-27. S. J. Russell and P. Norvig, Artificial Intelligence: A Modern Approach: Pearson Education/Prentice Hall, 2010. International Civil Aviation Organization (ICAO), "Unmanned Aircraft Systems (UAS). Cir 328 AN/190," United Nations, Montreal, Quebec, Canada2011. B. T. Clough, "Metrics, Schmetrics. How The Heck Do You Determine A UAV's Autonomy Anyway," Air Force Research Lab., Wright-Patterson AFB, OH.2002. R. Parasuraman, T. B. Sheridan, and C. D. Wickens, "A model for types and levels of human interaction with automation," Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on, vol. 30, pp. 286-297, 2000. U.S. Department of Defense, "Unmanned Aircraft Systems Roadmap (UAS), 2005-2030," 2005. H. M. Huang, "Autonomy Levels for Unmanned Systems (ALFUS) framework: Safety and application issues," in Performance Metrics for Intelligent Systems (PerMIS) Workshop, 2007, pp. 48-53. Y. Wang and J. Liu, "Evaluation methods for the autonomy of unmanned systems," Chinese Science Bulletin, vol. 57, pp. 3409-3418, 2012. C. C. Insaurralde and D. M. Lane, "Autonomy-assessment criteria for underwater vehicles," in Autonomous Underwater Vehicles (AUV), 2012 IEEE/OES, 2012, pp. 1-8. A. Doboli, D. Curiac, D. Pescaru, S. Doboli, W. Tang, C. Volosencu, et al., "Cities of the future: Employing wireless sensor networks for efficient decision making in complex environments," State University of New York, Tech. Rep, 2008. T. B. Curtin, D. M. Crimmins, J. Curcio, M. Benjamin, and C. Roper, "Autonomous underwater vehicles: trends and transformations," Marine Technology Society Journal, vol. 39, pp. 65-75, 2005. S. D. Prior, S. T. Shen, M. A. Erbil, M. Brazinskas, and W. Mielniczek, "HALO the Winning Entry to the DARPA UAVForge Challenge 2012," in Design, User Experience, and Usability. User Experience in Novel Technological Environments, ed: Springer, 2013, pp. 179-188. K. Nonami, F. Kendoul, S. Suzuki, W. Wang, and D. Nakazawa, Autonomous Flying Robots: Springer Japan, 2010. N. Michael, S. Shen, K. Mohta, Y. Mulgaonkar, V. Kumar, K. Nagatani, et al., "Collaborative mapping of an earthquakedamaged building via ground and aerial robots," Journal of Field Robotics, vol. 29, pp. 832-841, 2012. R. R. Murphy, E. Steimle, C. Griffin, C. Cullins, M. Hall, and K. Pratt, "Cooperative use of unmanned sea surface and 37
[27]
[28] [29] [30] [31] [32] [33] [34] [35] [36] [37]
[38] [39] [40]
[41] [42] [43] [44] [45] [46] [47] [48] [49]
[50] [51] [52] [53] [54] [55] [56]
[57] [58] [59] [60]
micro aerial vehicles at Hurricane Wilma," Journal of Field Robotics, vol. 25, pp. 164-180, 2008. D. Scaramuzza, M. C. Achtelik, L. Doitsidis, F. Friedrich, E. Kosmatopoulos, A. Martinelli, et al., "Vision-Controlled Micro Flying Robots: From System Design to Autonomous Navigation and Mapping in GPS-Denied Environments," Robotics & Automation Magazine, IEEE, vol. 21, pp. 26-40, 2014. R. Clothier, R. Walker, R. Baumeister, M. Brunig, J. Roberts, A. Duggan, et al., "The Smart Skies Project," Aerospace and Electronic Systems Magazine, IEEE, vol. 26, pp. 14-23, 2011. Paparazzi UAV. (2016). Welcome to Paparazzi UAV. Available: http://wiki.paparazziuav.org/wiki/Main_Page Pixhawk. (2016, 09/02). Pixhawk Autopilot. Available: https://pixhawk.org/modules/pixhawk V. Kumar and N. Michael, "Opportunities and challenges with autonomous micro aerial vehicles," The International Journal of Robotics Research, vol. 31, pp. 1279-1291, September 1, 2012 2012. F. Kendoul, "Survey of advances in guidance, navigation, and control of unmanned rotorcraft systems," Journal of Field Robotics, vol. 29, pp. 315-378, 2012. C. Goerzen, Z. Kong, and B. Mettler, "A Survey of Motion Planning Algorithms from the Perspective of Autonomous UAV Guidance," Journal of Intelligent and Robotic Systems, vol. 57, pp. 65-100, 2010/01/01 2010. L. Petricca, P. Ohlckers, and C. Grinde, "Micro- and Nano-Air Vehicles: State of the Art," International Journal of Aerospace Engineering, vol. 2011, 2011. L. Heng, D. Honegger, G. H. Lee, L. Meier, P. Tanskanen, F. Fraundorfer, et al., "Autonomous Visual Mapping and Exploration With a Micro Aerial Vehicle," Journal of Field Robotics, vol. 31, pp. 654-675, 2014. S. Weiss, M. W. Achtelik, S. Lynen, M. C. Achtelik, L. Kneip, M. Chli, et al., "Monocular Vision for Long-term Micro Aerial Vehicle State Estimation: A Compendium," Journal of Field Robotics, vol. 30, pp. 803-831, 2013. B. MacAllister, J. Butzke, A. Kushleyev, H. Pandey, and M. Likhachev, "Path planning for non-circular micro aerial vehicles in constrained environments," in Robotics and Automation (ICRA), 2013 IEEE International Conference on, 2013, pp. 3933-3940. I. Ferain, C. A. Colinge, and J.-P. Colinge, "Multigate transistors as the future of classical metal-oxide-semiconductor field-effect transistors," Nature, vol. 479, pp. 310-316, 11/17/print 2011. J. Bialkowski, S. Karaman, and E. Frazzoli, "Massively parallelizing the RRT and the RRT*," in Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on, 2011, pp. 3513-3518. A. Ratter, C. Sammut, and M. McGill, "GPU accelerated graph SLAM and occupancy voxel based ICP for encoder-free mobile robots," in Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on, 2013, pp. 540547. Intel. (2014, 26/09). ARK: Your Source for Intel Product Information. Available: http://ark.intel.com/ (2006, October) Computer Power User. (2002, December) Computer Power User. (2000) PC Magazine. A. R. Omondi and J. C. Rajapakse, FPGA implementations of neural networks vol. 365: Springer, 2006. A. Sutton. (2009, 16/11/2015). Beyond Physical: Solving High-end FPGA Design Challenges. [White Paper]. Available: https://www.synopsys.com/cgi-bin/proto/pdfdla/pdfr1.cgi?file=sbg-fpga-design-wp.pdf J. J. Rodriguez-Andina, M. J. Moure, and M. D. Valdes, "Features, Design Tools, and Application Domains of FPGAs," Industrial Electronics, IEEE Transactions on, vol. 54, pp. 1810-1823, 2007. E. Monmasson, L. Idkhajine, M. N. Cirstea, I. Bahri, A. Tisan, and M. W. Naouar, "FPGAs in Industrial Control Applications," Industrial Informatics, IEEE Transactions on, vol. 7, pp. 224-243, 2011. G. J. García, C. A. Jara, J. Pomares, A. Alabdo, L. M. Poggi, and F. Torres, "A survey on FPGA-Based sensor systems: towards intelligent and reconfigurable low-power sensors for computer vision, control and signal processing," Sensors, vol. 14, pp. 6247-6278, 2014. I. Kuon, R. Tessier, and J. Rose, "FPGA architecture: Survey and challenges," Foundations and Trends in Electronic Design Automation, vol. 2, pp. 135-253, 2008. K. Ian and J. Rose, "Measuring the Gap Between FPGAs and ASICs," Computer-Aided Design of Integrated Circuits and Systems, IEEE Transactions on, vol. 26, pp. 203-215, 2007. T. Merz and F. Kendoul, "Dependable Low-altitude Obstacle Avoidance for Robotic Helicopters Operating in Rural Areas," Journal of Field Robotics, vol. 30, pp. 439-471, 2013. A. Kelly and A. Stentz, "Rough Terrain Autonomous Mobility—Part 1: A Theoretical Analysis of Requirements," Autonomous Robots, vol. 5, pp. 129-161, 1998/05/01 1998. NASA. (2014, 05/01). Unmanned Aircraft Systems Integration in the National Airspace System [Electronics]. Available: https://www.nasa.gov/centers/armstrong/news/FactSheets/FS-075-DFRC.html E. Frew and T. Brown, "Networking Issues for Small Unmanned Aircraft Systems," Journal of Intelligent and Robotic Systems, vol. 54, pp. 21-37, 2009/03/01 2009. RTCA, "Minimum Aviation System Performance Standard for AMS(R)S Data and Voice Communications Supporting Required Communications Performance (RCP) and Required Surveillance Performance (RSP) in Procedural Airspace," Radio Technical Commission for Aeronautics DO-343, 2013. K. Dar, M. Bakhouya, J. Gaber, M. Wack, and P. Lorenz, "Wireless communication technologies for ITS applications [Topics in Automotive Networking]," Communications Magazine, IEEE, vol. 48, pp. 156-162, 2010. (January). ITU. Available: http://www.itu.int/. R. J. Kerczewski, J. D. Wilson, and W. D. Bishop, "Frequency spectrum for integration of unmanned aircraft," in Digital Avionics Systems Conference (DASC), 2013 IEEE/AIAA 32nd, 2013, pp. 6D5-1-6D5-9. D. Foty, B. Smith, S. Sinha, and M. Schroter, "Expanding wireless bandwidth in a power-efficient way: developing a viable mm-wave radio technology," in East-West Design & Test Symposium, 2013, 2013, pp. 1-6. 38
[61] [62]
[63] [64]
S. Bouabdallah, M. Becker, and R. Siegwart, "Autonomous miniature flying robots: coming soon! - Research, Development, and Results," Robotics & Automation Magazine, IEEE, vol. 14, pp. 88-98, 2007. R. J. Kerczewski, J. D. Wilson, and W. D. Bishop, "Parameter impact on sharing studies between UAS CNPC satellite transmitters and terrestrial systems," in Integrated Communication, Navigation, and Surveillance Conference (ICNS), 2015, 2015, pp. X2-1-X2-11. H. T. Friis, "A Note on a Simple Transmission Formula," Proceedings of the IRE, vol. 34, pp. 254-256, 1946. AT&T, "AT&T and Intel® to Test Drones on LTE Network," in AT&T and Intel Demo LTE Connected Drone in Intel’s
Drone Zone at Mobile World Congress in Barcelona, ed. Barcelona, Spain: AT&T Newsroom, 2016. [65] G. Warwick, "Cellular Technology, Challenges At Heart Of Small UAS Plans," in Aviation Week & Space Technology, ed, 2015, p. 21. [66] N. Goddemeier, K. Daniel, and C. Wietfeld, "Role-Based Connectivity Management with Realistic Air-to-Ground Channels for Cooperative UAVs," Selected Areas in Communications, IEEE Journal on, vol. 30, pp. 951-963, 2012. [67] T. S. Rappaport, S. Shu, R. Mayzus, Z. Hang, Y. Azar, K. Wang, et al., "Millimeter Wave Mobile Communications for 5G Cellular: It Will Work!," Access, IEEE, vol. 1, pp. 335-349, 2013. [68] M. N. Simic, "Vehicular ad hoc networks," in 2013 11th International Conference on Telecommunications in Modern Satellite, Cable and Broadcasting Services, TELSIKS 2013, 2013, pp. 613-618. [69] ISO 21217 (International Organisation for Standardisation), "Intelligent transport systems - Communications access for land mobiles (CALM) - Architecture," ed. Geneva: ISO International Organisation for Standardisation, 2010. [70] G. Baldini, S. Karanasios, D. Allen, and F. Vergari, "Survey of wireless communication technologies for public safety," Communications Surveys & Tutorials, IEEE, vol. 16, pp. 619-641, 2014. [71] V. Sarihan, J. Wen, G. Li, T. Koschmieder, R. Hooper, R. Shumway, et al., "Designing small footprint, low-cost, highreliability packages for performance sensitive MEMS sensors," in Electronic Components and Technology Conference, 2008. ECTC 2008. 58th, 2008, pp. 817-818. [72] R. J. Kerczewski, J. D. Wilson, and W. D. Bishop, "Assessing spectrum compatibility for beyond-line-of-sight UAS control and non-payload communications," in Integrated Communications, Navigation and Surveillance Conference (ICNS), 2014, 2014, pp. K2-1-K2-9. [73] G. Iapichino, C. Bonnet, O. Herrero, C. Baudoin, and I. Buret, "A mobile ad-hoc satellite and wireless mesh networking approach for public safety communications," in Signal Processing for Space Communications, 2008. SPSC 2008. 10th International Workshop on, 2008, pp. 1-6. [74] B. Scrosati and J. Garche, "Lithium batteries: Status, prospects and future," Journal of Power Sources, vol. 195, pp. 24192430, 5/1/ 2010. [75] M. Bronz, J.-M. Moschetta, P. Brisset, and M. Gorraz, "Towards a long endurance MAV," in EMAV 2009, European Micro Aerial Vehicle Conference and Flight Competition, Delft, Netherlands, 2009, pp. pp 241-254. [76] P. Brent, E. Byron, H. Kevin, L. Mark, F. Kendall, and W. J. Bowman, "Development of a Micro Air Vehicle for Maximum Endurance and Minimum Size," in 41st Aerospace Sciences Meeting and Exhibit, ed: American Institute of Aeronautics and Astronautics, 2003. [77] R. Wagner, N. Preschitschek, S. Passerini, J. Leker, and M. Winter, "Current research trends and prospects among the various materials and designs used in lithium-based batteries," Journal of Applied Electrochemistry, vol. 43, pp. 481-496, 2013. [78] S. A. Brandt, "Small UAV Design Development and Sizing," in Handbook of Unmanned Aerial Vehicles, K. P. Valavanis and G. J. Vachtsevanos, Eds., ed: Springer Netherlands, 2014, pp. 165-180. [79] C. Chun-Lung, C. Yie-Tone, L. You-Len, and L. Ruey-Hsun, "Optimal Driving Efficiency Design for the Single-Phase Brushless DC Fan Motor," Magnetics, IEEE Transactions on, vol. 46, pp. 1123-1130, 2010. [80] G. Joel and K. Matthew, "Development of the Black Widow Micro Air Vehicle," in 39th Aerospace Sciences Meeting and Exhibit, ed: American Institute of Aeronautics and Astronautics, 2001. [81] A. Mohamed, S. Watkins, R. Clothier, and M. Abdulrahim, "Influence of Turbulence on MAV Roll Perturbations," International Journal of Micro Air Vehicles, vol. 6, pp. 175-190, 09/01/ 2014. [82] L. W. Traub, "Range and Endurance Estimates for Battery-Powered Aircraft," Journal of Aircraft, vol. 48, pp. 703-707, 2011/03/01 2011. [83] M. R. Waszak, L. N. Jenkins, and P. Ifju, "Stability and control properties of an aeroelastic fixed wing micro aerial vehicle," AIAA paper, vol. 4005, p. 2001, 2001. [84] A. M. DeLuca, M. F. Reeder, J. A. Freeman, and M. V. Ol, "Flexible-and rigid-wing micro air vehicle: lift and drag comparison," Journal of aircraft, vol. 43, pp. 572-575, 2006. [85] H. Zhang, X. Zhong, J. C. Shaw, L. Liu, Y. Huang, and X. Duan, "Very high energy density silicide-air primary batteries," Energy & Environmental Science, vol. 6, pp. 2621-2625, 2013. [86] M. Armand and J. M. Tarascon, "Building better batteries," Nature, vol. 451, pp. 652-657, 02/07/print 2008. [87] G. Zhou, F. Li, and H.-M. Cheng, "Progress in flexible lithium batteries and future prospects," Energy & Environmental Science, vol. 7, p. 1307, 2014. [88] D. Anderson, "An Evaulation of the Current and Future Costs for Lithium-Ion Batteries for Use in Electrified Vehicle Powertrains," Masters Thesis, Nicholas School of the Environment, Duke University, 2009. [89] J. Zhang, Z. Xie, W. Li, S. Dong, and M. Qu, "High-capacity graphene oxide/graphite/carbon nanotube composites for use in Li-ion battery anodes," Carbon, vol. 74, pp. 153-162, 8// 2014. [90] A. P. Cohn, L. Oakes, R. Carter, S. Chatterjee, A. S. Westover, K. Share, et al., "Assessing the improved performance of freestanding, flexible graphene and carbon nanotube hybrid foams for lithium ion battery anodes," Nanoscale, vol. 6, pp. 4669-4675, 2014. 39
[91]
[92]
[93] [94] [95] [96] [97] [98] [99] [100] [101] [102]
[103] [104] [105] [106] [107]
[108] [109] [110] [111] [112]
[113] [114] [115] [116] [117]
[118] [119]
[120]
[121] [122]
O. Gohardani, M. C. Elola, and C. Elizetxea, "Potential and prospective implementation of carbon nanotubes on next generation aircraft and space vehicles: A review of current and expected applications in aerospace sciences," Progress in Aerospace Sciences, vol. 70, pp. 42-68, 10// 2014. J. F. M. Oudenhoven, R. J. M. Vullers, and R. van Schaijk, "A review of the present situation and future developments of micro-batteries for wireless autonomous sensor systems," International Journal of Energy Research, vol. 36, pp. 11391150, 2012. P. A. Plonski, P. Tokekar, and V. Isler, "Energy-efficient Path Planning for Solar-powered Mobile Robots*," Journal of Field Robotics, vol. 30, pp. 583-601, 2013. P. Tokekar, N. Karnad, and V. Isler, "Energy-optimal trajectory planning for car-like robots," Autonomous Robots, pp. 122, 2014/04/03 2014. M. J. Cutler, T. W. McLain, R. W. Beard, and B. Capozzi, "Energy harvesting and mission effectiveness for small unmanned aircraft," in AIAA Guidance, Navigation and Control Conference, 2010. A. Chakrabarty and J. W. Langelaan, "Energy maps for long-range path planning for small-and micro - UAVs," 2009. J. W. Langelaan, "Tree-based trajectory planning to exploit atmospheric energy," in American Control Conference, 2008, 2008, pp. 2328-2333. P. Lissaman, "Wind energy extraction by birds and flight vehicles," in AIAA Aerospace Sciences Meeting and Exhibit, 2005, pp. 15339-15351. C. White, S. Watkins, E. Lim, and K. Massey, "The Soaring Potential of a Micro Air Vehicle in an Urban Environment," International Journal of Micro Air Vehicles, vol. 4, pp. 1-14, 03/01/ 2012. J. W. Langelaan and N. Roy, "Enabling New Missions for Robotic Aircraft," Science, vol. 326, pp. 1642-1644, December 18, 2009 2009. C. White, E. W. Lim, S. Watkins, A. Mohamed, and M. Thompson, "A feasibility study of micro air vehicles soaring tall buildings," Journal of Wind Engineering and Industrial Aerodynamics, vol. 103, pp. 41-49, 4// 2012. S. Watkins, A. Mohamed, A. Fisher, R. Clothier, R. Carrese, and D. F. Fletcher, "Towards Autonomous MAV Soaring in Cities: CFD Simulation, EFD Measurement and Flight Trials," International Journal of Micro Air Vehicles, vol. 7, pp. 441-448, December 1, 2015 2015. A. Fisher, A. Mohamed, M. Elbanhawi, R. Clothier, S. watkins, R. Carrese, et al., "MAV soaring in Urban Environments," presented at the Australian Control Conference, Newcastle, Australia, 2016. A. Chakrabarty and J. W. Langelaan, "Flight path planning for uav atmospheric energy harvesting using heuristic search," in AIAA Guidance, Navigation and Controls Conference, Toronto, Canada, 2010. M. Bronz, J. M. Moschetta, P. Brisset, and M. Gorraz, "Towards a Long Endurance MAV," International Journal of Micro Air Vehicles, vol. 1, pp. 241-254, 12/01/ 2009. S. Baldis, H. Baldis, and J. Baldis, "Wireless Power Transmission System," USA Patent, 2006. M. C. Achtelik, J. Stumpf, D. Gurdan, and K. M. Doth, "Design of a flexible high performance quadcopter platform breaking the MAV endurance record with laser power beaming," in Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on, 2011, pp. 5166-5172. Y. Mulgaonkar and V. Kumar, "Autonomous charging to enable long-endurance missions for small aerial robots," in SPIE Defense+ Security, 2014, pp. 90831S-90831S-15. T. J. Nugent and J. T. Kare, "Laser power for uavs," Laser Motive White Paper-Power Beaming for UAVs, NWEN, 2010. T. Nayfeh, B. Fast, D. Raible, D. Dinca, N. Tollis, and A. Jalics, "High intensity laser power beaming architecture for space and terrestrial missions," 2011. A. Woodworth and J. Peverill, "Tethered aerial system for data gathering," ed: Google Patents, 2013. D. B. Wilson, A. H. Goktogan, and S. Sukkarieh, "Experimental validation of a drogue estimation algorithm for autonomous aerial refueling," in Robotics and Automation (ICRA), 2015 IEEE International Conference on, 2015, pp. 5318-5323. T. Chang and H. Yu, "Improving Electric Powered UAVs’ Endurance by Incorporating Battery Dumping Concept," 2014. L. Yangbo and D. J. Cappelleri, "Linear control design, allocation, and implementation for the Omnicopter MAV," in Robotics and Automation (ICRA), 2013 IEEE International Conference on, 2013, pp. 289-294. J. Ratti and G. Vachtsevanos, "Inventing a Biologically Inspired, Energy Efficient Micro Aerial Vehicle," Journal of Intelligent & Robotic Systems, vol. 65, pp. 437-455, 01 January 2012 2012. P. Segui-Gasco, Y. Al-Rihani, H.-S. Shin, and A. Savvaris, "A Novel Actuation Concept for a Multi Rotor UAV," Journal of Intelligent & Robotic Systems, vol. 74, pp. 173-191, 2014/04/01 2014. L. Bohwa, K. Sejin, P. Park, and K. Kim, "Active power management system for an unmanned aerial vehicle powered by solar cells, a fuel cell, and batteries," Aerospace and Electronic Systems, IEEE Transactions on, vol. 50, pp. 3167-3177, 2014. D. Verstraete, "On the energy efficiency of hydrogen-fuelled transport aircraft," International Journal of Hydrogen Energy, vol. 40, pp. 7388-7394, 2015. A. Gong, J. L. Palmer, G. Brian, J. R. Harvey, and D. Verstraete, "Performance of a hybrid, fuel-cell-based power system during simulated small unmanned aircraft missions," International Journal of Hydrogen Energy, vol. 41, pp. 1141811426, 2016. D. Verstraete, J. R. Harvey, and J. L. Palmer, "Hardware-in-the-loop simulation of fuel-cell-based hybrid-electrical UAV propulsion," in Proceedings of the 28th Congress of the International Council of the Aeronautical Sciences, September, 2012, pp. 23-28. A. Mohamed, K. Massey, S. Watkins, and R. Clothier, "The attitude control of fixed-wing MAVS in turbulent environments," Progress in Aerospace Sciences, vol. 66, pp. 37-48, 4// 2014. S. Chen, Y. Li, and N. M. Kwok, "Active vision in robotic systems: A survey of recent developments," The International 40
[123] [124] [125]
[126] [127] [128] [129] [130] [131] [132]
[133] [134] [135] [136] [137] [138] [139] [140] [141] [142] [143] [144] [145] [146] [147] [148] [149] [150]
[151] [152] [153] [154]
[155]
Journal of Robotics Research, vol. 30, pp. 1343-1377, 2011. J. D. Holmes, Wind loading of structures: CRC Press, 2001. J. Peterka, R. Meroney, and K. Kothari, "Wind flow patterns about buildings," Journal of Wind Engineering and Industrial Aerodynamics, vol. 21, pp. 21-38, 1985. J.-J. Kim and J.-J. Baik, "A numerical study of the effects of ambient wind direction on flow and dispersion in urban street canyons using the RNG< i> k–< i> ε turbulence model," Atmospheric Environment, vol. 38, pp. 3039-3048, 2004. G. Leuzzi and P. Monti, "Particle trajectory simulation of dispersion around a building," Atmospheric Environment, vol. 32, pp. 203-214, 1998. Y. Zhang, S. Arya, and W. Snyder, "A comparison of numerical and physical modeling of stable atmospheric flow and dispersion around a cubical building," Atmospheric Environment, vol. 30, pp. 1327-1345, 1996. A. D. Ferreira, A. C. M. Sousa, and D. X. Viegas, "Prediction of building interference effects on pedestrian level comfort," Journal of Wind Engineering and Industrial Aerodynamics, vol. 90, pp. 305-319, 2002. A. Mochida and I. Y. F. Lun, "Prediction of wind environment and thermal comfort at pedestrian level in urban area," Journal of Wind Engineering and Industrial Aerodynamics, vol. 96, pp. 1498-1527, 2008. W. Frost and A. M. Shahabi, "A field study of wind over a simulated block building," 1977. H. G.-C. Woo, J. A. Peterka, and J. E. Cermak, "Wind-tunnel measurements in the wakes of structures," 1977. K. E. W. Lim, S. Watkins, R. Clothier, R. Ladani, A. Mohamed, and J. L. Palmer, "Full-scale flow measurement on a tall building with a continuous-wave Doppler Lidar anemometer," Journal of Wind Engineering and Industrial Aerodynamics, vol. 154, pp. 69-75, 2016. A. Mohamed, M. Abdulrahim, S. Watkins, and R. Clothier, "Development and flight testing of a turbulence mitigation system for micro air vehicles," Journal of Field Robotics, 2015. D. E. J. Walshe, "Wind-Excited Oscillation of Structures," Her Majesty's Stationery Office, 1972. J. S. McCarley and C. D. Wickens, "Human factors concerns in UAV flight," University of Illinois at Urbana-Champaign Institute of Aviation, Aviation Human Factors Division, 2004. M. Orr, S. Rasmussen, E. Karni, and E. Blake, "Framework for developing and evaluating MAV control algorithms in a realistic urban setting," presented at the American Control Conference, Portland, OR USA, 2005. C. R. Galinski, "Gust Resistant Fixed Wing Micro Air Vehicle," Journal of Aircraft, vol. 43, pp. 1586-1588, 2006/09/01 2006. C. Galiński and R. Żbikowski, "Some problems of micro air vehicles development," Bulletin of the polish academy of sciences; technical sciences, vol. 55, pp. 91-98, 2007. P. Lissaman, "Effects of Turbulence on Bank Upsets of Small Flight Vehicles," presented at the 47th AIAA Aerospace Sciences Meeting including The New Horizons Forum and Aerospace Exposition, 2009. W. Shyy, H. Aono, S. K. Chimakurthi, P. Trizila, C.-K. Kang, C. E. Cesnik, et al., "Recent progress in flapping wing aerodynamics and aeroelasticity," Progress in Aerospace Sciences, vol. 46, pp. 284-327, 2010. P. Poksawat, L. Wang, and A. Mohamed, "Automatic tuning of attitude control system for fixed-wing unmanned aerial vehicles," IET Control Theory & Applications, 2016. F. Ruffier and N. Franceschini, "Optic flow regulation: the key to aircraft automatic guidance," Robotics and Autonomous Systems, vol. 50, pp. 177-194, 2005. F. Aubépart and N. Franceschini, "Bio-inspired optic flow sensors based on FPGA: Application to Micro-Air-Vehicles," Microprocessors and Microsystems, vol. 31, pp. 408-419, 2007. F. Kendoul, K. Nonami, I. Fantoni, and R. Lozano, "An adaptive vision-based autopilot for mini flying machines guidance, navigation and control," Autonomous Robots, vol. 27, pp. 165-188, 2009/10/01 2009. X. Dong, K. Yoshida, and T. A. Stoffregen, "Control of a virtual vehicle influences postural activity and motion sickness," Journal of Experimental Psychology: Applied, vol. 17, pp. 128-138, 2011. J. C. Zufferey and D. Floreano, "Fly-inspired visual steering of an ultralight indoor aircraft," Robotics, IEEE Transactions on, vol. 22, pp. 137-146, 2006. J. Kim and G. Brambley, "Dual optic-flow integrated navigation for small-scale flying robots," in In: Proceedings of the Australasian Conference on Robotics and Automation, 2007. J. Chahl, K. Rosser, and A. Mizutani, "Bioinspired optical sensors for unmanned aerial systems," in Proceedings of SPIE, 2011, p. 797503. A. Ollero and L. Merino, "Control and perception techniques for aerial robotics," Annual Reviews in Control, vol. 28, pp. 167-178, 2004. A. Mohamed, S. Watkins, R. Clothier, M. Abdulrahim, K. Massey, and R. Sabatini, "Fixed-wing MAV attitude stability in atmospheric turbulence—Part 2: Investigating biologically-inspired sensors," Progress in Aerospace Sciences, vol. 71, pp. 1-13, 2014. A. Mohamed, S. Watkins, A. Fisher, M. Marino, K. Massey, and R. Clothier, "A Feasibility Study of Bio-Inspired WingSurface Pressure Sensing for Attitude Control of MAVs," Journal of Aircraft, vol. 152, pp. 827-838, 2014. A. Mohamed, M. Abdulrahim, S. Watkins, and R. Clothier, "Development and Flight Testing of a Turbulence Mitigation System for MAVs," Journal of Field Robotics, pp. 1-22, 2014. D. Yeo, E. M. Atkins, L. P. Bernal, and W. Shyy, "Aerodynamic Sensing for a Fixed Wing UAS Operating at High Angles of Attack," presented at the AIAA Atmospheric Flight Mechanics Conference, Minneapolis, Minnesota, 2012. C. J. Blower, W. Lee, and A. M. Wickenheiser, "The development of a closed-loop flight controller with panel method integration for gust alleviation using biomimetic feathers on aircraft wings," presented at the ASME Conference on Smart Materials, Adaptive Structures and Intelligent Systems, Pheonix, AZ, 2011. L. Kleeman and R. Kuc, "Sonar Sensing," in Springer Handbook of Robotics, B. Siciliano and O. Khatib, Eds., ed: 41
[156]
[157] [158] [159] [160] [161] [162]
[163] [164] [165] [166] [167] [168] [169] [170]
[171] [172] [173] [174] [175] [176] [177] [178] [179] [180]
[181] [182] [183]
[184] [185] [186] [187]
[188]
Springer Berlin Heidelberg, 2008, pp. 491-519. J. Xuecheng, I. Ladabaum, F. Degertekin, S. Calmes, and B. T. Khuri-Yakub, "Fabrication and characterization of surface micromachined capacitive ultrasonic immersion transducers," Microelectromechanical Systems, Journal of, vol. 8, pp. 100-114, 1999. R. Siegwart, I. R. Nourbakhsh, and D. Scaramuzza, Introduction to Autonomous Mobile Robots: Mit Press, 2011. A. Elfes, "Sonar-based real-world mapping and navigation," Robotics and Automation, IEEE Journal of, vol. 3, pp. 249265, 1987. A. Zelinsky, "Mobile robot map making using sonar," Journal of Robotic Systems, vol. 8, pp. 557-577, 1991. S. Thrun, D. Fox, W. Burgard, and F. Dellaert, "Robust Monte Carlo localization for mobile robots," Artificial Intelligence, vol. 128, pp. 99-141, 5// 2001. J. Steckel and H. Peremans, "BatSLAM: Simultaneous Localization and Mapping Using Biomimetic Sonar," PLoS ONE, vol. 8, p. e54076, 2013. G. Flores, S. Zhou, R. Lozano, and P. Castillo, "A Vision and GPS-Based Real-Time Trajectory Planning for a MAV in Unknown and Low-Sunlight Environments," Journal of Intelligent & Robotic Systems, vol. 74, pp. 59-67, 2014/04/01 2014. F. Endres, J. Hess, J. Sturm, D. Cremers, and W. Burgard, "3-D Mapping With an RGB-D Camera," Robotics, IEEE Transactions on, vol. 30, pp. 177-187, 2014. J.-H. Kim, J. Starr, and B. Lattimer, "Firefighting Robot Stereo Infrared Vision and Radar Sensor Fusion for Imaging through Smoke," Fire Technology, pp. 1-23, 2014/06/05 2014. S. Vidas and S. Sridharan, "Hand-held monocular SLAM in thermal-infrared," in Control Automation Robotics & Vision (ICARCV), 2012 12th International Conference on, 2012, pp. 859-864. FLIR Commercial Vision Systems. (2014, 26/11/14). Cooled versus uncooled cameras for long range surveillance. 2. Available: http://www.flir.com/uploadedfiles/Eurasia/MMC/Tech_Notes/TN_0005_EN.pdf R. Sabatini, M. A. Richardson, A. Gardi, and S. Ramasamy, "Airborne laser sensors and integrated systems," Progress in Aerospace Sciences, vol. 79, pp. 15-63, 2015. Performance Standards for Light-Emitting Products, U.S. Food and Drug Administration (FDA) 1040, 2014. Hokuyo. (2015, 12/01). Scanning range finders. Available: https://www.hokuyo-aut.jp/02sensor/ SICK. (2015, 12/01). 2D laser scanners. Available: http://www.sick.com/au/enus/home/products/product_portfolio/laser_measurement_systems/Pages/indoor_laser_measurement_technolo gy.aspx P. Corke, Robotics, Vision and Control: Fundamental Algorithms in MATLAB: Springer, 2011. J. Johnson, "Analysis of image forming systems," presented at the Image Intensifier Symposium, Warfare Electrical Engineering Department, U.S. Army Research and Development Laboratories, Fort Belvoir, Virgina, USA, 1958. D. Floreano, R. Pericet-Camara, S. Viollet, F. Ruffier, A. Bruckner, R. Leitel, et al., "Miniature curved artificial compound eyes," Proc Natl Acad Sci U S A, vol. 110, pp. 9267-72, Jun 4 2013. M. W. Achtelik, S. Lynen, S. Weiss, M. Chli, and R. Siegwart, "Motion- and Uncertainty-aware Path Planning for Micro Aerial Vehicles," Journal of Field Robotics, vol. 31, pp. 676-698, 2014. P. Corke, J. Lobo, and J. Dias, "An Introduction to Inertial and Visual Sensing," The International Journal of Robotics Research, vol. 26, pp. 519-535, June 1, 2007 2007. H. Durrant-Whyte and T. Henderson, "Multisensor Data Fusion," in Springer Handbook of Robotics, B. Siciliano and O. Khatib, Eds., ed: Springer Berlin Heidelberg, 2008, pp. 585-610. Y. Mimeault, D. Cantin, and P. Gallant, "LED object detection system and method combining complete reflection traces from individual narrow field-of-view channels," ed: Google Patents, 2012. D. Cantin, Y. Mimeault, and P. Gallant, "Lighting system and method for illuminating and detecting object," ed: Google Patents, 2010. E. Skafidas, F. Zhang, B. Yang, B. Wicks, Z. Liu, C. Ta, et al., "A 60-GHz transceiver on CMOS," in Proc. Microw. Photon. ointly Held With the 2008 Asia-Pacific Microw. Photon. Conf.(MWP2008/APMP2008), 2008, pp. 306-309. S. T. Nicolson, P. Chevalier, B. Sautreuil, and S. P. Voinigescu, "Single-Chip W-band SiGe HBT Transceivers and Receivers for Doppler Radar and Millimeter-Wave Imaging," IEEE Journal of Solid-State Circuits, vol. 43, pp. 22062217, 2008. M. Li, R. J. Evans, E. Skafidas, and B. Moran, "Radar-on-a-chip (ROACH)," in Radar Conference, 2010 IEEE, 2010, pp. 1224-1228. C. Niclass, A. Rochas, P.-A. Besse, and E. Charbon, "Design and characterization of a CMOS 3-D image sensor based on single photon avalanche diodes," Solid-State Circuits, IEEE Journal of, vol. 40, pp. 1847-1854, 2005. C. Veerappan, J. Richardson, R. Walker, D.-U. Li, M. W. Fishburn, Y. Maruyama, et al., "A 160× 128 single-photon image sensor with on-pixel 55ps 10b time-to-digital converter," in Solid-State Circuits Conference Digest of Technical Papers (ISSCC), 2011 IEEE International, 2011, pp. 312-314. M. W. Fishburn and E. Charbon, "System Tradeoffs in Gamma-Ray Detection Utilizing SPAD Arrays and Scintillators," IEEE Transactions on Nuclear Science, vol. 57, pp. 2549-2557, 2010. V. Braitenberg, Vehicles: Experiments in synthetic psychology Cambridge, Mass: MIT Press, 1984. V. J. Lumelsky and A. A. Stepanov, "Dynamic path planning for a mobile automaton with limited information on the environment," Automatic Control, IEEE Transactions on, vol. 31, pp. 1058-1063, 1986. Z. Sam, C. Mark, M. Ankur, G. Greg, M. Derek, R. Badri, et al., "Experimental Study of Gust Effects on Micro Air Vehicles," in AIAA Atmospheric Flight Mechanics Conference, ed: American Institute of Aeronautics and Astronautics, 2010. E. Galceran and M. Carreras, "A survey on coverage path planning for robotics," Robotics and Autonomous Systems, vol. 42
[189] [190]
[191] [192] [193] [194] [195] [196] [197] [198] [199] [200] [201]
[202] [203]
[204] [205]
[206] [207] [208] [209] [210] [211] [212] [213] [214] [215] [216] [217] [218] [219] [220]
61, pp. 1258-1276, 12// 2013. Y. Kwangjin, S. Keat Gan, and S. Sukkarieh, "A Gaussian process-based RRT planner for the exploration of an unknown and cluttered environment with a UAV," Advanced Robotics, vol. 27, pp. 431-443, 2013/04/01 2013. S. Shen, N. Michael, and V. Kumar, "Stochastic differential equation-based exploration algorithm for autonomous indoor 3D exploration with a micro-aerial vehicle," The International Journal of Robotics Research, vol. 31, pp. 1431-1444, October 1, 2012 2012. H. Ruijie, A. Bachrach, and N. Roy, "Efficient planning under uncertainty for a target-tracking micro-aerial vehicle," in Robotics and Automation (ICRA), 2010 IEEE International Conference on, 2010, pp. 1-8. V. N. Dobrokhodov, I. I. Kaminer, K. D. Jones, and R. Ghabcheloo, "Vision-based tracking and motion estimation for moving targets using small UAVs," in American Control Conference, 2006, 2006, p. 6 pp. N. Guenard, T. Hamel, and R. Mahony, "A Practical Visual Servo Control for an Unmanned Aerial Vehicle," Robotics, IEEE Transactions on, vol. 24, pp. 331-340, 2008. H. Myung, J. Kuffner, and T. Kanade, "Efficient Two-phase 3D Motion Planning for Small Fixed-wing UAVs," in Robotics and Automation, 2007 IEEE International Conference on, 2007, pp. 1035-1041. A. Chakrabarty and J. Langelaan, "UAV flight path planning in time varying complex wind-fields," in American Control Conference (ACC), 2013, 2013, pp. 2568-2574. L. De Filippis, G. Guglieri, and F. Quagliotti, "Path Planning Strategies for UAVS in 3D Environments," Journal of Intelligent & Robotic Systems, vol. 65, pp. 247-264, 2011. S. Chakravorty and S. Kumar, "Generalized Sampling-Based Motion Planners," Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, vol. 41, pp. 855-866, 2011. R. Huq, G. K. I. Mann, and R. G. Gosine, "Behavior-modulation technique in mobile robotics using fuzzy discrete event system," Robotics, IEEE Transactions on, vol. 22, pp. 903-916, 2006. H. Seraji and A. Howard, "Behavior-based robot navigation on challenging terrain: A fuzzy logic approach," Robotics and Automation, IEEE Transactions on, vol. 18, pp. 308-321, 2002. P. Wu, R. Clothier, D. Campbell, and R. Walker, "Fuzzy Multi-Objective Mission Flight Planning in Unmanned Aerial Systems," in Computational Intelligence in Multicriteria Decision Making, IEEE Symposium on, 2007, pp. 2-9. J. G. Hurdus and D. W. Hong, "Behavioral programming with hierarchy and parallelism in the DARPA urban challenge and robocup," in Multisensor Fusion and Integration for Intelligent Systems, 2008. MFI 2008. IEEE International Conference on, 2008, pp. 503-509. D. Ferguson, C. Baker, M. Likhachev, and J. Dolan, "A reasoning framework for autonomous urban driving," in Intelligent Vehicles Symposium, 2008 IEEE, 2008, pp. 775-780. M. Montemerlo, J. Becker, S. Bhat, H. Dahlkamp, D. Dolgov, S. Ettinger, et al., "Junior: The Stanford Entry in the Urban Challenge," in The DARPA Urban Challenge. vol. 56, M. Buehler, K. Iagnemma, and S. Singh, Eds., ed: Springer Berlin Heidelberg, 2009, pp. 91-123. E. Besada-Portas, L. de la Torre, J. M. de la Cruz, A. de, x, and B. s-Toro, "Evolutionary Trajectory Planner for Multiple UAVs in Realistic Scenarios," Robotics, IEEE Transactions on, vol. 26, pp. 619-634, 2010. K. Jong-Hwan, H. Ji-Hyeong, K. Ye-Hoon, C. Seung-Hwan, and K. Eun-Soo, "Preference-Based Solution Selection Algorithm for Evolutionary Multiobjective Optimization," Evolutionary Computation, IEEE Transactions on, vol. 16, pp. 20-34, 2012. M. Dorigo, M. Birattari, and T. Stutzle, "Ant colony optimization," Computational Intelligence Magazine, IEEE, vol. 1, pp. 28-39, 2006. C. D. Gelatt, Jr., S. Kirkpatrick, and M. P. Vecchi, "Optimization by simulated annealing," Science, vol. 220, p. 671, 1983/05/13 1983. D. Heckerman, D. Geiger, and D. Chickering, "Learning Bayesian networks: The combination of knowledge and statistical data," Machine Learning, vol. 20, pp. 197-243, 1995/09/01 1995. A. Gardi, R. Sabatini, and S. Ramasamy, "Multi-objective optimisation of aircraft flight trajectories in the ATM and avionics context," Progress in Aerospace Sciences, vol. 83, pp. 1-36, 2016. J. Canny, J. Reif, B. Donald, and P. Xavier, "On the complexity of kinodynamic planning," in Foundations of Computer Science, 1988., 29th Annual Symposium on, 1988, pp. 306-316. J.-C. Latombe, Robot Motion Planning: Springer, 1990. T. Asano, T. Asano, L. Guibas, J. Hershberger, and H. Imai, "Visibility-polygon search and euclidean shortest paths," in Foundations of Computer Science, 1985., 26th Annual Symposium on, 1985, pp. 155-164. R. A. Brooks and T. Lozano-Perez, "A subdivision algorithm in configuration space for findpath with rotation," Systems, Man and Cybernetics, IEEE Transactions on, vol. SMC-15, pp. 224-233, 1985. J. Canny, "A Voronoi method for the piano-movers problem," in Robotics and Automation. Proceedings. 1985 IEEE International Conference on, 1985, pp. 530-535. J. M. Keil and J.-R. Sack, "Minimum decompositions of polygonal objects," Computational Geometry, pp. 197-216, 1985. A. Stentz, "Optimal and efficient path planning for unknown and dynamic environments," International Journal of Robotics and Automation, vol. 10, pp. 89-100, // 1995. S. Koenig, M. Likhachev, and D. Furcy, "Lifelong Planning A∗," Artificial Intelligence, vol. 155, pp. 93-146, 5// 2004. M. Likhachev, D. Ferguson, G. Gordon, A. Stentz, and S. Thrun, "Anytime search in dynamic graphs," Artificial Intelligence, vol. 172, pp. 1613-1643, 9// 2008. O. Khatib, "Real-Time Obstacle Avoidance for Manipulators and Mobile Robots," International Journal of Robotics Research, vol. 5, pp. 90-98, Spr 1986. Y. Koren and J. Borenstein, "Potential field methods and their inherent limitations for mobile robot navigation," in 43
[221] [222] [223] [224] [225]
[226] [227]
[228] [229] [230] [231] [232] [233] [234] [235] [236] [237]
[238] [239] [240] [241] [242]
[243] [244] [245]
[246] [247] [248] [249]
[250]
Robotics and Automation, 1991. Proceedings., 1991 IEEE International Conference on, 1991, pp. 1398-1404 vol.2. M. Elbanhawi and M. Simic, "Sampling-Based Robot Motion Planning: A Review," Access, IEEE, vol. 2, pp. 56-77, 2014. L. E. Kavraki, P. Svestka, J. C. Latombe, and M. H. Overmars, "Probabilistic roadmaps for path planning in highdimensional configuration spaces," Robotics and Automation, IEEE Transactions on, vol. 12, pp. 566-580, 1996. S. M. LaValle, "Rapidly-exploring random trees: A new tool for path planning," Iowa State University TR 98-11, 1998. M. Elbanhawi, M. Simic, and R. Jazar, "Randomized Bidirectional B-Spline Parameterization Motion Planning," Intelligent Transportation Systems, IEEE Transactions on, vol. PP, pp. 1-1, 2015. I. K. Nikolos, K. P. Valavanis, N. C. Tsourveloudis, and A. N. Kostaras, "Evolutionary algorithm based offline/online path planner for UAV navigation," Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, vol. 33, pp. 898-912, 2003. E. Magid, D. Keren, E. Rivlin, and I. Yavneh, "Spline-Based Robot Navigation," in Intelligent Robots and Systems, 2006 IEEE/RSJ International Conference on, 2006, pp. 2296-2301. E. Koyuncu and G. Inalhan, "A probabilistic B-spline motion planning algorithm for unmanned helicopters flying in dense 3D environments," in Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on, 2008, pp. 815-821. Y. Kwangjin and S. Sukkarieh, "3D smooth path planning for a UAV in cluttered natural environments," in Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on, 2008, pp. 794-800. Y. Kwangjin and S. Sukkarieh, "An Analytical Continuous-Curvature Path-Smoothing Algorithm," Robotics, IEEE Transactions on, vol. 26, pp. 561-568, 2010. M. Elbanhawi and M. Simic, "Randomised kinodynamic motion planning for an autonomous vehicle in semi-structured agricultural areas," Biosystems Engineering, vol. 126, pp. 30-44, 10// 2014. S. LaValle, Planning Algorithms: Cambridge University Press, 2006. L. Jingru and K. Hauser, "An empirical study of optimal motion planning," in Intelligent Robots and Systems (IROS 2014), 2014 IEEE/RSJ International Conference on, 2014, pp. 1761-1768. M. Elbanhawi, M. Simic, and R. Jazar, "Receding horizon lateral vehicle control for pure pursuit path tracking," Journal of Vibration and Control, May 22, 2016 2016. M. Elbanhawi, M. Simic, and R. Jazar, "Improved manoeuvring of autonomous passenger vehicles: Simulations and field results," Journal of Vibration and Control, October 20, 2015 2015. M. Elbanhawi, M. Simic, and R. Jazar, "The Role of Path Continuity in Lateral Vehicle Control," Procedia Computer Science, vol. 60, pp. 1289-1298, // 2015. S. M. LaValle and J. J. Kuffner, "Randomized Kinodynamic Planning," The International Journal of Robotics Research, vol. 20, pp. 378-400, May 1, 2001 2001. F. Andert, F. Adolf, L. Goormann, and J. Dittrich, "Mapping and path planning in complex environments: An obstacle avoidance approach for an unmanned helicopter," in Robotics and Automation (ICRA), 2011 IEEE International Conference on, 2011, pp. 745-750. M. Kothari and I. Postlethwaite, "A Probabilistically Robust Path Planning Algorithm for UAVs Using Rapidly-Exploring Random Trees," Journal of Intelligent & Robotic Systems, vol. 71, pp. 231-253, 2013/08/01 2013. M. W. Achtelik, S. Weiss, M. Chli, and R. Siegwart, "Path planning for motion dependent state estimation on micro aerial vehicles," in Robotics and Automation (ICRA), 2013 IEEE International Conference on, 2013, pp. 3926-3932. M. Likhachev and D. Ferguson, "Planning Long Dynamically Feasible Maneuvers for Autonomous Vehicles," The International Journal of Robotics Research, vol. 28, pp. 933-945, 1st of August 2009. H. Cover, S. Choudhury, S. Scherer, and S. Singh, "Sparse Tangential Network (SPARTAN): Motion planning for micro aerial vehicles," in Robotics and Automation (ICRA), 2013 IEEE International Conference on, 2013, pp. 2820-2825. T. Peter, H. Jason, W. Matthew, S. Greg, T. Marc, R. Matthew, et al., "A System for 3D Autonomous Rotorcraft Navigation in Urban Environments," in AIAA Guidance, Navigation and Control Conference and Exhibit, ed: American Institute of Aeronautics and Astronautics, 2008. S. Scherer, S. Singh, L. Chamberlain, and M. Elgersma, "Flying Fast and Low Among Obstacles: Methodology and Experiments," The International Journal of Robotics Research, vol. 27, pp. 549-574, 2008. H. Gabriel, W. Steven, and T. Claire, "Quadrotor Helicopter Trajectory Tracking Control," in AIAA Guidance, Navigation and Control Conference and Exhibit, ed: American Institute of Aeronautics and Astronautics, 2008. D. Jennifer, P. Mario, W. Brenton, and K. Ondrej, "Development of a Modified Voronoi Algorithm for UAV Path Planning and Obstacle Avoidance," in AIAA Guidance, Navigation, and Control Conference, ed: American Institute of Aeronautics and Astronautics, 2012. I. D. Cowling, O. A. Yakimenko, J. F. Whidborne, and A. K. Cooke, "Direct Method Based Control System for an Autonomous Quadrotor," Journal of Intelligent & Robotic Systems, vol. 60, pp. 285-316, 2010. Y. Zhang, J. Chen, and L. Shen, "Real-time trajectory planning for UCAV air-to-surface attack using inverse dynamics optimization method and receding horizon control," Chinese Journal of Aeronautics, vol. 26, pp. 1038-1056, 2013. T. Merz and F. Kendoul, "Dependable Low‐altitude Obstacle Avoidance for Robotic Helicopters Operating in Rural Areas," Journal of Field Robotics, vol. 30, pp. 439-471, 2013. S. Ross, N. Melik-Barkhudarov, K. S. Shankar, A. Wendel, D. Dey, J. A. Bagnell, et al., "Learning monocular reactive UAV control in cluttered natural environments," in Robotics and Automation (ICRA), 2013 IEEE International Conference on, 2013, pp. 1765-1772. M. Nieuwenhuisen, D. Droeschel, M. Beul, and S. Behnke, "Obstacle Detection and Navigation Planning for Autonomous Micro Aerial Vehicles," in Unmanned Aircraft Systems (ICUAS), 2014 International Conference on, 2014, pp. 10401047. 44
[251] [252] [253] [254] [255] [256] [257]
F. Belkhouche and B. Bendjilali, "Reactive path planning for 3-D autonomous vehicles," Control Systems Technology, IEEE Transactions on, vol. 20, pp. 249-256, 2012. P. Cheng, "Sampling-based motion planning with differential constraints," 3198947 Ph.D., Department of Computer Science, University of Illinois at Urbana-Champaign, Ann Arbor, 2005. T. Fraichard and H. Asama, "Inevitable collision states — a step towards safer robots?," Advanced Robotics, vol. 18, pp. 1001-1024, 2004/01/01 2004. D. Mellinger, N. Michael, and V. Kumar, "Trajectory generation and control for precise aggressive maneuvers with quadrotors," The International Journal of Robotics Research, vol. 31, pp. 664-674, April 1, 2012 2012. T. Hebecker, R. Buchholz, and F. Ortmeier, "Model-Based Local Path Planning for UAVs," Journal of Intelligent & Robotic Systems, vol. 78, pp. 127-142, 2015. P. Panyakeow and M. Mesbahi, "Deconfliction Algorithms for a Pair of Constant Speed Unmanned Aerial Vehicles," Aerospace and Electronic Systems, IEEE Transactions on, vol. 50, pp. 456-476, 2014. Federal Aviation Administration (FAA), "FAA fact sheet: FAA Forecast Fact Sheet Fiscal Years 2016 – 2036," Federal Aviation Administration, Washington DC, USA2016.
45
