UAVS IN WISAR Towards Using UAVs in

To appear Interaction Studies 10(3) 2009

UAVs in WiSAR 1

Running head: UAVS IN WISAR

Towards Using UAVs in Wilderness Search and Rescue: Lessons from Field Trials

Michael A. Goodrich, Bryan S. Morse, Cameron Engh Brigham Young University

Joseph L. Cooper University of Texas Austin

Julie A. Adams Vanderbilt University


UAVs in WiSAR 2

Abstract Wilderness Search and Rescue (WiSAR) is the process of finding and assisting persons who are lost in remote wilderness areas. Because such areas are often rugged or relatively inaccessible, searching for missing persons can take huge amounts of time and resources. Camera-equipped mini-Unmanned Aerial Vehicles (UAVs) have the potential for speeding up the search process by enabling searchers to view aerial video of an area of interest while closely coordinating with nearby ground searchers. In this paper, we report on lessons learned by trying to use UAVs to support WiSAR. Our research methodology has relied heavily on field trials involving searches conducted under the direction of practicing search and rescue personnel but using simulated missing persons. Lessons from these field trials include the immediate importance of seeing things well in the video, the field need for defining and supporting various roles in the search team, role-specific needs like supporting systematic search by providing a visualization tool to represent the quality of the search, and the on-going need to better support interactions between ground and video searchers. Surprisingly to us, sophisticated autonomous search patterns were less critical than we anticipated, though advances in video enhancement and visualizing search progress, as well as ongoing work to model the likely location of a missing person, open up the possibility of closing the loop between UAV path-planning, search quality, and the likely location of a moving missing person.


UAVs in WiSAR 3

Towards Using UAVs in Wilderness Search and Rescue: Lessons from Field Trials

Introduction Wilderness Search and Rescue (WiSAR) is the process of finding and assisting lost persons in wilderness settings. Such settings include forests, deserts, mountains, and other remote areas where humans explore and recreate. Each year thousands of hours and hundreds of thousands of dollars are spent looking for those who get lost in such remote areas. Because of the large areas and often rugged terrain associated with WiSAR work, hand-launched mini-Unmanned Aerial Vehicles (UAVs) can be used to provide near-ground aerial video to trained searchers to help detect and localize cues of a missing person’s location without incurring the expense or risk of involving manned aircraft. Introducing camera-equipped mini-UAVs into the WiSAR domain requires an understanding of real WiSAR situations and problems so that technology can be developed that provides real help to real searchers, rather than technology that is largely an intellectual exercise for researchers and scientists. A key element of developing technology that supports WiSAR is the use of field trials. In our work, a field trial involves placing a simulated missing person in a wilderness area, creating a story that includes realistic information about the point last seen and other search-critical information, deploying the UAV technology to perform a search, and dispatching ground searchers to evaluate cues if and when they are located. Our field trials have typically involved 10–15 researchers and a representative from Utah County Search and Rescue. Although field trials often include use of immature designs and technology, we have found it counterproductive to “fly” a technology for the first time during a field trial. Thus, supplementing field trials are regular flight tests that involve 2–4 researchers with the primary goal of making sure that a


UAVs in WiSAR 4

particular technology or application works. Although we refer to these as “flight tests,” they build on relatively mature UAV technologies and usually focus on user interface designs, computer vision algorithms, or systematic search algorithms rather than flight per se. In addition to flight tests, we frequently conduct user experiments to validate that technologies positively impact the intended use scenarios. The impact of flight tests, user experiments, and field trials on the technology development cycle is illustrated in Figure 1. In this paper, we describe lessons learned from a series of field trials and flight tests conducted between January 2005 and August 2008. Lessons from these field trials include the immediate importance of seeing things well in the video, the field need for defining and supporting various roles in the search team, role-specific needs like supporting systematic search by providing a visualization tool to represent the quality of the search, and the on-going need to better support interactions between ground and video searchers. Surprisingly to us, sophisticated autonomous search patterns were less critical than we anticipated, though advances in video enhancement and visualizing search progress, as well as ongoing work to model the likely location of a missing person, open up the possibility of closing the loop between UAV path-planning, search quality, and the likely location of a moving missing person. The paper is organized as follows. After presenting a brief overview of related work, we summarize two surprising lessons that emerged from trying to use a real UAV in a simulated search. We then present both a consensus-type and a qualitative analysis of field trials, leading to discussions of technologies that have been developed as a direct result of this analysis. These discussions include both reviews of previously published technologies as well as new technologies. We then summarize future work and present conclusions. Related Work In the interest of space, we reference prior literature reviews for more complete discussions (Goodrich et al., 2008; Adams et al., 2008), and restrict the literature in this paper to more recent work or holes in previous literature reviews. Additionally, we note that the bulk of the references in


UAVs in WiSAR 5

this paper are distributed throughout the paper rather than concentrated in this section. Recent work that is most similar to the presented work is the use of mini-rotocraft UAVs to evaluate damage caused by hurricanes and other natural disasters (Murphy et al., 2008). Similarly, there are currently efforts to improve systematic search using UAVs with the goal of efficiently gathering information on scales relevant to WiSAR (Schmale, Dingus, & Reinholts, 2008; Wong, Bourgault, & Furukawa, 2005; Bourgault, Furukawa, & Durrant-Whyte, 2003). Future work identified in this paper includes fusing information from air and ground sources, which should build from similar work on fusing information from humans, UAVs, and ground robots (Kaupp, Douillard, Ramos, Makarenki, & Upcroft, 2007). Naturally, much of the UAV-based work builds from lessons learned in ground-based work, including fielded efforts in urban search and rescue (Casper & Murphy, 2003) and the growing literature on such efforts for both search (Heth & Cornell, 1998; Behavior Characteristics from the Search and Rescue Society of British Colombia, n.d.) and military applications (Voyles & Choset, 2008; Hill & Bodt, 2007). The ability to even consider using mini-UAVs in WiSAR builds on the excellent work of many engineers and scientists, including those who have directly contributed to the autonomy and path-planning algorithms that we use (Beard et al., 2005; Quigley, Goodrich, & Beard, 2004; Nelson, Barber, McLain, & Beard, 2006). Additionally, this work benefits directly from seminal work in human factors of UAVs (Cooke, Pringle, Pederson, & Connor, 2006; Tao, Tharp, Zhang, & Tai, 1999) and the cognition of human navigation (Wang & Spelke, 2000). Lessons from Field Trials Formal field trials were conducted in March 2005, July 2006, October 2006 (two scenarios), September 2007, and May 2008. Each field trial occurred far away from airports and flight paths. With one exception, each field trial was conducted in an area with limited vegetation (sage brush, native grasses, and juniper trees) and with moderately undulating terrain (with changes in terrain less than the maximum rate with which the UAV can change altitude). The exception, the September 2007 field trial, occurred in an area with dramatic terrain slopes and vegetation consisting of a mix of relatively dense scrub oak and native grasses. During each search, a safety officer (a) monitored both the ground searchers and the people


UAVs in WiSAR 6

at the base camp to ensure that safe practices were followed and (b) watched for manned aircraft in the search area. Ground searchers had access to an ATV for one search but were on foot or bicycle for the other scenarios. For each scenario, the UAV maintained a height-above-ground of less than 100 meters. For each field trial, the simulated missing person was placed less than 1.5 miles from the UAV’s launch point, yielding a search area of less than seven square miles. The search scenarios were constructed by a student researcher who had two years experience as a search and rescue volunteer. Scenarios were based on previous lost person histories, but modified to comply with points of interest around the launch point such as campgrounds, trails, roads, etc. Initial field trials used a 3D simulated missing person such as an inflatable child’s toy or a CPR training mannequin, but subsequent field trials simply used clothing. The simulated missing person was positioned at least 30 minutes before the search team arrived, and the search team received information only from the scenario (not from the people who placed the simulated missing person). Occasionally, objects were placed near the simulated missing person, such as a bicycle, blanket, backpack, or food containers. With the exception of the May 2008 field trial in which a professor acted as supervisor, a member of Utah County Search and Rescue played the role of the incident commander (IC) in each field trial. When the search team arrived, they set up a temporary base camp with laptop computers, a generator, and a nylon shelter. After establishing base camp, the search team received a scenario brief and asked questions. Each scenario included correct and incorrect information. Correct information included details of the point last seen and direction of travel that were consistent with the true location of the simulated missing person. Incorrect information included the color of clothing and items that the missing person was thought to be carrying. The scenarios were constructed so that the missing person was unlikely to continue moving in order to isolate the search radius to a small area. After the briefing, the IC created a search plan, the UAV was launched, and the search began. Four search scenarios concluded successfully within six hours from launch time, and three scenarios resulted in failures since the missing person was not found during the time allotted.


UAVs in WiSAR 7

“Surprise” Lessons from the Field This section (a) describes what we anticipated would happen in a search and (b) contrasts what actually happens on a typical search to what we expected. We titled this section “Surprise” Lessons because our initial understanding of the process was naive and needed to change; we were surprised at some of the things that were important in real search. We are including it because both scholarly and informal presentations of this work often generate comments similar to our initial understanding of search “in the wild.” Lesson 1: The (Un)Importance of Search Patterns. We believed that the search pattern would have the greatest impact on the success of the search. This belief was based on two naive assumptions. First, it was assumed that detecting an object in the video would be easier than it is in practice. Second, it assumed that the UAV would stay aloft for a longer period of time than it could in practice. We address each of these assumptions. Prior to the field trials, the UAV systems, which were developed by the BYU MAGICC lab, had experienced extensive flight tests. These flight tests produced a recommended height above ground of 60 to 100 meters – no less than 60 meters to give enough slack to deal with trees and small variations in terrain, and no more than 100 meters so that a human form could be discerned in the video (Goodrich et al., 2008). The UAVs are small, are lightweight, and must continually move as they are fixed-wing. These factors create difficulties including the following: • Video is jittery, making it difficult to interpret features in the image, • Objects do not stay in the video for very long, making it difficult to detect objects, and • The UAV makes frequent turns, causing the corresponding video to spin and rotate, making it difficult to localize an object in a map and communicate its location to ground searchers. Simply put, the initial field trial failed largely because people could not find objects in the video even though the video was of acceptable resolution. It was not only a matter of the camera “seeing” the missing person, but also of the searcher seeing the objects of interest in the images.


UAVs in WiSAR 8

Field trials used UAVs with flight times between 45 and 105 minutes, thus the useful flight time was 30–90 minutes (the remaining time was for launch and landing, with extra time built in for safety). This limited flight time means that only a small area can be covered relative to the entire search area. Additionally, to increase the likelihood that an object of interest was discovered in the video, the UAV orbited a focal point while the focal point slowly traced a path over the ground that could support detection. Furthermore, if a potential object of interest was detected, it was necessary to gather additional images to determine if a ground crew should be dispatched to investigate the object. This further limited the amount of ground “covered” by the UAV. Although we believe that the search pattern is important, the challenges of difficult video detection plus the limitations on flight time meant that the search pattern was initially and surprisingly of far less concern than other practical aspects of search. Lesson 2: Coordinating UAV and Ground Searchers. Prior to initial field trials, we assumed that the UAV operator would fly a search pattern that would optimally cover the ground, that someone watching the video would detect and localize potential objects of interest, and that these potential objects of interest would be queued and passed to ground searchers who would go to the most likely object of interest and hopefully find the simulated missing person. This assumed a coordination between video analysts and ground searchers that was naive. This approach was naive because there are many situations in which close, real-time coordination is required between the base camp and the ground searchers. During the first field trial, a bright object was spotted in the video. A ground search team was dispatched to the general area of the object without having seen any aerial images containing the object. Communication between the UAV team and the ground searchers occurred via radios and cell phones, so the approximate GPS coordinates were communicated over the radio. The ground team was unable to determine exactly what the UAV team was referring to, so it became necessary to communicate landmark information. Eventually, the UAV was flown so that the ground team was visible, and “go left” or “go north” directions were provided by the video analysts. This


UAVs in WiSAR 9

example illustrates that queuing objects of interest prior to dispatching ground searchers may be viable for some search conditions (see (Goodrich, Cooper, et al., 2007)), but some searches may require close communication between air and ground searchers. Close, real-time coordination may be required due to the different kinds of information available from the air and ground perspectives. Some other examples illustrate this point. • An object about the size of a soda can or water bottle was found in the video. Ground searchers determined that it was a soda can, but noted that it was too old and weathered to have been discarded by the missing person and was irrelevant to the search. • A human-sized and shaped object was found in the video. Ground searchers determined that it was a mannequin used in martial arts training that had been used by careless party-goers as a target for shotgun practice and was irrelevant to the search. • A bright object was found in the video. Ground searchers were dispatched to the general location and located a dark bicycle reportedly used by the missing person. The dark bicycle was not easily visible from the air but was from the ground. In contrast, the bright object was part of the missing person’s clothing and was easily detected from the air but difficult to find from the ground. The second scenario of the second field trial used a different search paradigm, one where ground searchers were the primary drivers of the search instead of the UAV. The ground searchers simulated the process of tracking the missing person’s footprints. The UAV circled the ground searchers’ path while finding potential objects of interest and effectively increasing the ground searchers’ field of view. Although some irrelevant objects were detected that caused the ground searchers to deviate from the path, eventually the missing person was found from the air, even after the ground searchers lost the trail. The complementary types of information1 from the air and ground, the challenging communications, and the potential ways of coordinating air and ground searchers spurred work to understand how UAVs can positively impact the existing WiSAR processes. This work formally analyzed the existing WiSAR processes (Adams et al., 2008), identified several possible air-ground coordination


UAVs in WiSAR 10

paradigms (Goodrich, Cooper, et al., 2007), and focused on developing technologies to simultaneously support air and ground searchers under the direction of the person managing the search2 . Development Priorities from Initial Field Trials This section first presents how a consensus-based method was used to decide which technologies were most important. The 2006–2007 field trials collected rankings from seven participants who identified important and high priority technologies. The participants included faculty, students, and a representative from Utah County Search and Rescue3 . Each person participated in multiple field trials, and then subjectively ranked ten different technologies from highest to lowest in importance and priority. Importance indicates whether the technology is essential for a successful search. Priority is a judgment of which technologies should receive attention in the near future. Ten different technologies were ranked from zero to nine, with zero being the least important and nine the most important. The technologies are • HW: reliable UAV flight hardware, communications, and basic autonomy. • UI: user friendly operator interface. • Waypts: algorithms for autonomously generating waypoints. • StabVid: operator interfaces that present stable video. • IntGUI: operator interfaces that integrate maps and video. • HAG: autonomy for maintaining height above ground. • Proc: efficient coordination processes between operator, searchers, and IC. • ImEn: enhanced imagery to highlight visual signs. • OffSrch: operator interfaces that allow video to be searched offline. • Gim: a gimballed camera. We performed a nonparametric statistical analysis of the rankings to find a consensus between participants who judged the usefulness of the technologies. Although each judge was potentially biased, these biases were spread approximately evenly across the technologies because each judge focused on a


UAVs in WiSAR 11

different research topic. Since judges did not see the rankings of others and biases are approximately spread across the judges, we assume that each judge ranked technologies independently of all others. The non-parametric analysis used a strengthened form of majority rule that requires 60% of the population to make the same judgement. Let the test statistic N be the number of judges that rank technology A lower than technology B, and let q be the probability that a judge, randomly drawn from the population, will rank A higher than B. N is distributed according to a binomial distribution with seven samples and parameter q. Given the null hypothesis threshold, q ≥ 0.40, the probability that either six or seven randomly selected judges will rank A higher than B by chance is P(N ≥ 6) = 0.1586. Thus, if six or seven judges rank A higher than B, then we reject the null hypothesis that A is not more important (is not higher priority) than B with a confidence level of 0.1586. In the interest of space, a summary of the analysis is included, with details available in a technical report (Goodrich, Quigley, et al., 2007). The most important technologies were hardware, the ability to enhance raw video, the ability to maintain height above ground, and the ability to coordinate the search between various team members. Other technologies were not considered important given deficiencies in the important technologies. These include a user-friendly operator interface, support for autonomously generating waypoints for an efficient search path, the ability to perform offline search, and an operator interface that integrates map and video. The highest development priorities include image enhancement, an efficient integration of the UAV into the WiSAR team, and support for height above ground maintenance. Useful trends emerge when the importance and priority rankings are combined. The first two rows of Table 1 list the number of technologies that are significantly less important (lower priority) than the listed technology; e.g., the priority value of 5 for StVid means that, according to the nonparametric analysis, StVid was a significantly higher priority than 5 other technologies. Taking the product of the first two rows gives the last row of Table 1; this row indicates which technologies are essential but not developed. Two types of technologies are evaluated most highly: improved video (consisting of stabilized video and image enhancement) and improved process for using the UAV with WiSAR personnel. It is important to


UAVs in WiSAR 12

note that this quantitative analysis supports the qualitative observations discussed as “surprise” lessons. Given these results, Section summarizes research that was performed to address the development priorities. This research is presented in review form since it has previously appeared in the literature. Development Priorities from Most Recent Field Trial The most recent field trial, conducted in May 2008, followed the pattern from prior trials,. Unlike several prior field trials, there were very few hardware difficulties. The UAV flew within 45 minutes of arriving on scene and a systematic search pattern was performed. The field trial used a 4.5 foot, fixed wing UAV purchased from Procerus Technologies with their Virtual Cockpit user interface. The video analyst used the video mosaicking software, and WiSAR practices identified in the cognitive task analysis were followed. This field trial benefited from these developments, allowing us to identify new technologies that are on the critical path to full field deployment. The first search phase lasted approximately 30 minutes and ended unsuccessfully. This search phase used a gimbaled camera, but the camera settings “washed-out” information making it very difficult to detect objects of interest. The UAV was retrieved, the batteries changed, a fixed-mount camera (with correct optical settings) replaced the gimbaled camera, and the UAV was relaunched. A lawnmower search pattern was selected and the missing person was located in about 40 minutes. Following the field trial, we performed a qualitative analysis of the trial to help identify future research and development. Eight participants completed a questionnaire evaluating different aspects of the trial. Each participant responded to questions, including (a) surprises and disappointments to (b) evaluations of technologies are or are not ready for field deployment. Responses to each question were categorized by field trial participants who did not complete the evaluation. Category summarizations were then discussed in a face-to-face debriefing with most of the evaluators. From the results, three positive statements can be made about what currently is field ready. First, the UAV can be quickly deployed and flown. Second, the mosaic software is useful for detecting (but not localizing) objects. Third, the lawnmower search pattern is useful on flat terrain with limited vegetation.


UAVs in WiSAR 13

However, there are two significant obstacles to widespread field deployment. First, current technologies will not work over very rugged terrain with dense vegetation. Second, communications need to be improved, including electronic communication technologies and team member communication protocols. Finally, there is a real need for improved user interfaces including: • Interfaces for communicating between ground searchers and base station, including sharing images, GPS tracking of searchers/UAV. • Interfaces that integrate multiple information sources (video, mosaic, satellite imagery, UAV position, ground searcher locations, terrain, search quality/coverage) and support coordination between multiple people at base station (UAV operator, real-time video analysts, offline video analysts, IC). • Interfaces to control mosaicked video, including scrubbing, zooming, pausing, resuming, localizing. Although we were already working on some of these problems, these lessons serve to set priorities for future work. One key technology listed above is the development of a user interface that integrates multiple information sources. Sections and , respectively, report on user interfaces that (a) support the UAV operator by fusing multiple information sources and (b) support the IC by displaying the quality of coverage. These interfaces have been used in flight tests and partially evaluated in user experiments, but neither has yet been used in a field trial4 . Prior Development This section reviews two contributions that resulted from the consensus-based method for evaluating technologies from the initial field trials. Video Mosaicking One of the highest priorities identified was the the need to make the UAV search video more usable. To provide both stabilization and increased opportunity for observation, we use a temporally local mosaic (Morse et al., 2008). In computer vision the term mosaic is used to describe the result of “stitching”


UAVs in WiSAR 14

together multiple frames of video from a moving camera viewing a static scene to provide a single larger-area image (Szeliski, 2006). Over time, the alignment error between frames typically accumulates, making it impractical to align all the frames of a single search video sequence into a single large image without highly accurate telemetry or offline global optimization. However, we have found that creating small mosaics of only the last few seconds of video is sufficient to expand the observer’s spatiotemporal window into the search area, providing both increased opportunity for detection and an increased sense of relative spatial relationships. These temporally local mosaics are updated continuously as new frames are received and older frames scroll off the physical display area (Figure 2). The Figure 2 mosaic is from the most recent field trial and shows the simulated missing person on the left of the display (red shirt, blue jeans). Although only visible in a few video frames, the persistent presentation in the mosaicked display allowed for successful detection. User studies, reported in (Morse et al., 2008), demonstrated that these temporally local mosaics can increase search detection by 43% (p < 0.0001). There was an accompanying modest increase in false positives, but further analysis showed that these additional false positives were not generally due to artifacts of the display but were confounding objects in the environment that were likewise more easily detected. Subjectively, the users found temporally local mosaicked displays easier to use. They also had a more accurate sense of (less often overestimated) their own performance, which can be critical for search and rescue. Cognitive Task Analysis. A cognitive task analysis focused on analyzing the existing WiSAR response without UAVs was performed to understand how to coordinate the WiSAR team members (Adams et al., 2008). The analysis viewed the purposes that the human responders serve and the functions they fulfill as system components. This perspective provided a focus on the responders’ tasks, roles, and responsibilities and how they will be modified by introducing UAVs. Two analysis methods were applied: Goal-Directed Task Analysis (GDTA) (Endsley, Bolte, & Jones, 2003) and a partial Cognitive Work Analysis, namely Work Domain


UAVs in WiSAR 15

Analysis (WDA) (Vicente, 1999) and Control Task Analysis (ConTA)(Vicente, 1999). An explanation of the differences and contributions of each method is beyond the scope of this paper, but we emphasize that the combination of these techniques permitted the identification of important lessons that directly impacted the UAV system development. A key GDTA result produced was identifing the four primary WiSAR search techniques (hasty, constrained, high priority, and exhaustive). The UAV development team was unaware of these search techniques, reflecting the naive understanding previously discussed. Our recent efforts have shifted exhaustive and complete coverage searches (Quigley et al., 2004) to priority searches. The GDTA also identified that searches require considerable human judgment, especially as new missing person signs are collected; this lesson is relevant to Section . The WDA results informed both the UAV-enabled WiSAR team organization and the tactics including the operator interfaces reported in Sections and . The WDA clearly identified two key WiSAR response subsystems: information acquisition and information analysis. These subsystems provide a natural personnel division into roles with information acquisition and analysis responsibilities. The UAV operator is responsible for maintaining the UAV’s flight and is thus responsible for information acquisition. The UAV video analyst focuses on analyzing video to identify pertinent information and to filter out irrelevant information. The third role is a ground searcher, who is responsible for obtaining information unavailable in video (information acquisition), thus providing a complementary ability to interpret and refine clues to the missing person’s location (information analysis). The UAV development team had hoped that the UAV operator and video analyst roles could be combined, but field trials verify that this is not yet possible (Goodrich, Cooper, et al., 2007; Cooper & Goodrich, 2008). The need for the UAV operator, video analyst, and ground searchers to work as a team reinforce the qualitative “surprise” lessons reported previously. The WDA also distinguished between the information acquisition and information analysis subsystems and identified the possibility of conducting real-time, on-line information acquisition and


UAVs in WiSAR 16

off-line information analysis. This result, combined with the ConTA results, contributed to a descriptive framework of the temporally and spatially distributed UAV-enabled WiSAR roles (Goodrich, Cooper, et al., 2007). The ConTA resulted in a general understanding of the WiSAR mission timing and the information flow during the search. A very important result from an information management perspective is that the search is essentially an iterated refinement of the probability describing the missing person’s likely location. The ConTA provided insight into the role that probability refinement plays and its importance during the search, which was not fully understood. This information resulted in a Bayesian framework (Goodrich et al., 2008) that formalizes a search as an optimization problem, maximizing the evidence accumulation rate. The ConTA also provided further insight into UAV technical search team organization in that the iterated probability refinement requires a fourth human role, the UAV technical search specialty manager. This individual is responsible for planning and revising the search process as evidence accumulates or changes. This search manager may be the IC during a small search or the mission manager during a large search. Coverage: Supporting the UAV Operator This section and the next section present new research that was performed partly as a result of the lessons learned from the initial field trials, partly from problems identified by the task analysis described in the previous section, and partly from ongoing qualitative observations from flight tests and field trials. A user interface is being developed that can be classified as an “augmented virtuality” interface (Milgram & Kishino, 1994). The interface is virtual because it fuses terrain information, a map, and satellite imagery into a model displayed as a virtual 3D world. This virtual world is augmented with real information from the UAV including video and telemetry that are added as, respectively, (a) real-time video projected onto the map surface and (b) as a virtual UAV displaying the real UAV’s position and pose as well as the relative pose of the gimbaled camera. The virtual 3D world displays information from a variety of perspectives — an ability that has allowed us to identify perspectives that are most appropriate


UAVs in WiSAR 17

for the different search types identified by the GDTA. Figure 3(a) illustrates the main interface features, showing the UAV and video from a “chase” perspective (this perspective will be described shortly). The 3D synthetic terrain model is created using publicly available USGS digital elevation data and satellite imagery or topographic maps. The operator can annotate terrain model areas or plan flight search patterns. The model can highlight potential terrain dangers and provide context for UAV and video information. High detail video is necessary for information extraction from the imagery. Showing a large terrain area is desirable since it can provide greater awareness for long term planning. Integrating the two is important for helping the operator when extracting geo-referencing data from the imagery. Geo-referencing, which is the process of associating imagery and telemetry information with physical coordinates, potentially allows the operator to report information to the IC or ground searchers in a common reference frame. To control the UAV, the augmented virtuality interface uses a “carrot and camera” control metaphor (see Figure 3(a)). The “carrot” is an icon that follows the mouse and creates a floating waypoint to guide the UAV. If the UAV reaches the carrot, it first flies over and then circles the point (e.g. loiters). When the mouse moves by more than a specified amount, the interface updates the waypoint location and the UAV “follows the carrot”. A similar model provides the camera control, allowing one to click on a location in the terrain model and the gimbaled camera points there. Projecting a 3D synthetic terrain model to a 2D display requires a concept of a virtual camera, which defines the frame of reference and perspective from which the model is viewed (Blinn, 1988). The virtual camera’s behavior affects what information is available and how easily it can be understood (Wickens, Olmos, Chudy, & Davenport, 1997). Different perspectives are desirable for different flying tasks because they communicate different information and support different cognitive models (Alexander, Wickens, & Merwin, 2005; John & Cowen, 1999). The chase perspective (Figure 3(a)) refers to a reference frame wherein the origin is defined by the UAV, the upward axis is defined by gravity, the forward axis represents the UAV’s heading, and the third


UAVs in WiSAR 18

axis is orthogonal to the first two. The virtual camera is behind and perhaps slightly above the UAV and is oriented to focus on the UAV. A track-up perspective (Figure 3(b)) uses the same reference frame, but with the virtual camera looking down from above the UAV so that the UAV’s flying direction is upward on the display and when the UAV turns, the visual effect is the terrain rotating in the opposite direction. A north-up perspective (Figure 3(c)) still uses a UAV centered coordinate system, but all three axes are defined with respect to the terrain: up, north, and east. As with the track-up, the virtual camera looks down on the UAV, but the UAV turns within the display and the terrain remains in a constant, north-up orientation. A full-map perspective (not shown) uses the same axes as north-up, but defines the coordinate system with respect to the terrain instead of the UAV. The virtual camera looks downward, but from a sufficiently high vantage point in order to see all or most of the search area. These perspectives can be contrasted with a pilot’s perspective (Figure 3(d)) that uses a reference frame built completely around the UAV (i.e., one axis aligned with the wing, one through the top of the UAV, and one through the nose). The virtual camera is located in the UAV looking out the nose. Additional perspectives such as a side view are possible, and may be needed for extremely rugged terrain where it may be necessary to point the gimbaled camera to the side and follow terrain contours (Goodrich et al., 2008). A previous paper presented results from a simulation-based experiment where the effect of the display perspective on the ability to perform a search was compared (Cooper & Goodrich, 2008). The experiment compared the usefulness of four virtual perspectives (chase, north-up, track-up, and full-map perspective) in finding hidden target objects (simulating prior but uncertain knowledge of the missing person’s likely location) in the midst of several distracting objects. Both the target and the distractor objects were distributed according to the same probability distribution, and the experiment with humans considered three different distributions: uniform, gaussian, and path. The path distribution simulated a series of cues left by a mobile missing person. Because the distractor objects followed an informative prior distribution, these objects could be used to localize where the target objects might be found. The most interesting result, related to this paper, is that the “right” perspective depends on the task


UAVs in WiSAR 19

being done. The chase perspective was best when the targets/distractors were distributed along a path; this probably occurred because the chase perspective anchored reasoning about the UAV control as the relative UAV pose with respect to the objects of interest, thus allowing the operator to reactively track the distribution. In contrast, the north-up perspective was best when targets/distractors were distributed according to a uniform or a gaussian distribution; this probably occurred because the north-up display anchored the search in a fixed map that supported exhaustive search. Importantly, the traditional map-based perspective resulted in the poorest results. This occurred because the UAV operator had to translate what was seen in the map to what was seen in the video, since these were presented on separate screens. The other perspectives better allowed the operator to integrate video with map information, thus supporting more effective searches. Interestingly, prior results reported that the track-up display offers significant benefits over other perspectives (Wickens & Prevett, 1995). Our results did not support such prior work, probably because prior work used interfaces that controlled the UAV employing more traditional UAV controls such as moving the joystick left to bank the UAV left. The carrot-and-camera control metaphor is anchored in the UAV’s relative position in the world and does not benefit from UAV-centered control. In summary, our prior work suggests that the best virtual perspective depends on the kind of search being performed. We anticipate using the augmented virtuality interface in a field trial in the next few months. Coverage: Supporting the Search Manager This section presents the second new technology that was developed as a result of lessons learned in analyzing field trials. Augmented virtuality allows information to be fused into a usable form for the UAV operator, but this operator is not the only one responsible for interpreting information from multiple sources. The mission manager, who is responsible for planning, assessing, and replanning search steps can also benefit from reliably fused information. One of the mission manager’s key responsibilities is translating what is


UAVs in WiSAR 20

known about what the searchers have seen into a plan to gather more information. This section presents the notion of a coverage map and an associated “see-ability” metric that supports mission planning given realistic expectations of what can and cannot be detected in UAV provided video. The goal is for the operator to instantaneously understand what portions of the searched terrain have been seen by the camera and how well the recording captures ground detail. Because the recording is video there arise two potential measures of see-ability: • Instantaneous See-ability - How well is terrain seen by a single frame of video? • Collective See-ability - How well is terrain seen by all frames of video collectively? Instantaneous See-ability The goal of instantaneous see-ability is to determine for each point of the target search area the resolution with which that point is seen by the camera in a single video frame. This requires that each frame be georegistered to the underlying terrain, for which the limited accuracy of the UAV provided telemetry is often insufficient. Using computer vision techniques similar to those in (Wildes et al., 2001), georegistration is performed by refining pose estimates from the UAV’s telemetry using terrain models, corresponding reference aerial or satellite imagery, and the camera’s video feed (Engh, 2008). Once each video frame is georeferenced to the corresponding terrain, we compute the resolution with which each point in the terrain is seen by that frame. We do this by sampling the terrain using a regular grid of points. For each grid point in the current video frame, we compute the inverse of the projected pixel size to give a pixels-per-meter metric for see-ability. This depends on the viewing distance d, the camera’s focal length f , and the viewing angle of incidence with respect to perpendicular θ: f cos θ d

! (1)

If the camera’s focal length remains constant, we may treat it as a consistent scaling and omit it. We then define a minimum distance α from the camera at which a direct perpendicular view is chosen to be 100% seeable. The see-ability then approaches zero asymptotically as the distance from the camera increases and


UAVs in WiSAR 21

the angle of incidence becomes more oblique: cos θ 1 + d/α

! (2)

Since the see-ability of each point i differs within each frame j, and the see-ability of a single point i varies across different frames j, we calculate an instantaneous see-ability value S i j for each point/frame combination. Denoting di j as the distance from point i to the camera position for frame j, ~ni as the terrain unit normal at point i, and ~vi j as the unit vector from point i to frame j’s camera position, and explicitly including visibility determination, the instantaneous seeability is    ~ni ·~vi j     1+di j /α if point i is visiible in frame j S ij =       0 otherwise

(3)

The resulting see-ability measurement can be scaled to the range [0,255] and used to shade the terrain area map, as in Fig. 4. A single UAV pose is used to compute the distance and angle to every point visible from the video frame. Validation of Instantaneous See-ability Prior work framed WiSAR as a Bayesian problem that decomposes the problem into separate coverage and detection probabilities; the former represents the probability that the UAV gathers imagery containing an object of interest and the latter represents the likelihood that a human observer detects the object in an image (Goodrich et al., 2008). If the see-ability metric correlates with the probability that a human observer detects an object of interest, then see-ability can be very useful to an IC when refining the search. The validity of the instantaneous see-ability metric was tested in an initial user study designed to measure how closely a user’s ability to detect detail matches the metric’s prediction. Experiment Design. The experimental set up was: • A virtual scene was created for an area covered by a UAV flight. • Individual UAV poses were selected at random from the flight.


UAVs in WiSAR 22

• The virtual terrain was augmented by laying a marker in a randomly selected location on the terrain surface. • The potential marker locations were limited to regions that were visible from the selected UAV pose. • Each marker was a 3D object approximately the size of a human figure and with typical clothing colors; all markers were identical. Each participant was shown a series of images or “frames” from the scene. Each frame was dynamically generated by selecting a telemetry value from the list of telemetry entries for the UAV and recreating the camera pose within the virtual world. The selected frames should therefore be representative of the entire flight. The frame was randomly designated as either a marker frame or a control frame. The marker frame marker was rendered in its predefined position. No marker was rendered for a control frame. The same set of frames was used for all participants; however, the frame order presentation was random. The experiment included 50 marker and 30 control frames. Participants were trained with written instructions and examples. A small demographic survey and a self-evaluation of the frequency of computer usage were completed before the experimental test. Each frame was displayed for two seconds. During the two seconds, the participant scanned the scene and attempted to determine if a marker was visible. If the participant saw the marker they pressed the space bar to indicate detection. Pressing the space bar again toggled their selection. This was useful for detecting accidental presses. After two seconds the screen displayed white noise and the participant rested for one second. During the rest period, the participant could press the space bar to toggle their selection. The intent of the simple binary selector was to measure if the object was seen, not whether they could select the object. By using a still frame, it was possible to pinpoint what distance, angles and other factors contributed to instantaneous see-ability. Thus, the detection rates are compared with predictions made by instantaneous see-ability as a function of distance and angle. In order to increase ecological validity, an image of a real person (the “hiker”) represented the


UAVs in WiSAR 23

marker. For each marker, we placed the hiker at the marker’s position, oriented to match the local terrain, and overlaid on top of the corresponding terrain. Figure 5(a) shows a normal frame without the hiker, while 5(b) shows a frame with the hiker rendered onto the terrain. The most difficult challenge was the selection of valid and representative pose information to construct the displayed frames. We wanted to select the telemetry pose information from actual UAV telemetry rather than simply select convenient poses. This was accomplished by iterating through all poses from a long and varied flight. The distance and angle parameters used in Equations 1–3 are key elements of instantaneous see-ability, so we evaluated three possible parameters for each, corresponding to best-case and worst-case scenarios as well as in-between. Marker frames were generated for all nine combinations of distances and angles. Experiment Findings. Participants quickly found that the LCD monitor’s viewing angle affected how well they discerned the marker. The image backgrounds were mostly earth tones and greens while the marker was an image of a hiker wearing blue-jeans and a red jacket. By moving their head to the right or left they discovered that the monitor’s contrast changed, increasing the marker’s saliency. This resulted in detection rates higher than the pilot study. A second, more influential finding was the role the terrain color played in detection. The terrain imagery consisted of vegetation, which was dark green, and earth, a light tan. The marker was much more difficult to locate when placed over vegetation. A third finding was that the control images did not come into play as expected. In fact, only one person mistakenly claimed to have detected a hiker in a control image. The resulting data showed a 0.56 correlation coefficient with high statistical significance (p < 0.0001) between the instantaneous see-ability prediction and the measured detection rates. This represents a moderately strong correlation, especially when considering the other factors that can affect


UAVs in WiSAR 24

detection rates. The results also showed a strictly proportional relationship between the see-ability prediction and measured detection rates, with a negligible y-intercept on the fitted linear relationship. Future Work There are several important areas of future work that need to be performed to make reliable UAV-enabled WiSAR possible. We first present an overview of several areas of future work and then discuss in detail one key area for future work. General Areas One important area for future work is improved access to video in the field. Field trials suggest that it would probably be useful to be able to blend online video enhancement for real-time search with offline scrubbing/searching of video to refine information and localize objects. Additionally, there is a need for both ground searchers and UAV-based searchers to share information through a common user interface paradigm. Moreover, the ability to model missing persons as the basis for an optimal or near-optimal Bayesian search is important. Interestingly, the ability to model missing persons, and to receive input from ground and UAV-based searchers, could benefit from seeability measures that integrate multiple sources of information. As such seeability estimates improve, it may be possible to “close the loop” between the seeability estimate and the UAV’s path planning yielding plans that optimally “see” the world with respect to the likely location of the missing person and the prior observations from air and ground searchers. Building on such search, it should be possible to include a human in the loop for this search, allowing mixed initiative planning of the UAV and ground search plans. Because seeability is a key element of such future work, it is instructive to explore future work on collective seeability in more detail. Key Area: Collective Seeability While instantaneous see-ability is relatively straightforward to compute, its usefulness is limited in a video-based application. Collective see-ability, illustrated in Figure 6, can depend on many factors


UAVs in WiSAR 25

including: • How many unique directions has the location been seen from? • What is the collective quality of the observations of a location? Unique Directions. In many cases, it may be insufficient to see a point well from a single viewing direction. While the instantaneous see-ability calculations include terrain visibility and account for cases where parts of the terrain may not be visible due to other terrain features (e.g., a ridge blocking terrain behind it), there may be points that are invisible due to other environmental aspects. As a result, some terrain points may be occluded from some viewing directions but fully visible from others. For example, if the missing person is resting against one side of a large rock, seeing that rock repeatedly from the other side does not provide additional information. To discover the hidden individual the UAV must capture that point from multiple viewing directions. The approach that generated Figure 6(c) accounted for this dissimilarity of views by considering the uniqueness of the viewing direction for each observation of a single terrain point. Repeated observations from the same direction do not add new information, observations from directions similar to those of previous directions add slightly more information, while observations from significantly different directions add the most information. We can do this discretely by extending the angular-partitioning approach and determining the uniqueness of each cell by computing the shortest angular distance to the nearest used neighboring cell. This approach can be modified to a continuous form by determining the compass direction θi j for which point i is viewed by frame j then finding the nearest similar viewing direction from which that point was observed in other frames. The uniqueness Ui j is computed by taking the angular difference (modulo 2π) from this most-similar viewing direction and normalizing it to the range [0, 1]: mink, j θi j − θik Ui j = π

(4)

Collective Quality. To combine the instantaneous see-ability of a point from multiple video frames, we model see-ability as a subjective probability. Every individual instantaneous measurement can be


UAVs in WiSAR 26

treated as the subjective probability that a participant is able to detect a target or point of interest in the frame. Although the true detection probability is unknown, we assume that a linear increase of spatial resolution results in a linear increase in the subjective detection probability. We combine multiple viewings using these subjective probabilities and basic probability to calculate the probability of detection on any frame: Si = 1 −

Y

1 − S ij

(5)

j

Note that while the Multiple Viewings metric is not used directly, it is implicit in Eq. 5 as repeated observations of the same point always increase the computed cumulative detection probability. In order to incorporate the Unique Angle metric we simply factor in the computed Ui j term. Eq. 5 considered S i to be the subjective detection probability in an observation. Now we alter our perspective slightly and consider S i j Ui j to be the subjective probability of an observation adding information to the detection. This new computation can estimate the subjective probability that any observation adds detection information to the search: Si = 1 −

Y

1 − S i j Ui j

(6)

j

Summary Although Wilderness Search and Rescue seems to be an ideal problem for mini-UAVs, field trials indicate that there are a number of serious obstacles for using UAVs in practical WiSAR. Field trials can be used to identify deficiencies in current technologies and to set development priorities. Using field trials, supplemented by flight tests and user experiments, we have made progress toward practical UAV-enabled WiSAR, though there is still much to be done to use UAVs in realistic search settings over varying terrains and conditions. Four specific results were presented: video mosaicking improves detection, analyzing existing WiSAR processes leads to better UAV-enabled WiSAR technologies, interfaces that fuse multiple information sources benefit from flexible virtual perspectives, and the quality of the search coverage can partially be captured using computer vision techniques in a see-ability metric.


UAVs in WiSAR 27

Author Biographies Michael A. Goodrich is an associate professor in the Computer Science Department at Brigham Young University. He received is Ph.D. in Electrical and Computer Engineering from Brigham Young University in 1996, completed a post-doctoral research position at Nissan Cambridge Basic Research in Cambridge, Massachusetts from 1996-1998, and joined the Computer Science faculty at BYU in 1998. Bryan S. Morse is an associate professor in the Department of Computer Science at Brigham Young University. He received his Ph.D. in Computer Science from the University of North Carolina at Chapel Hill in 1995. Cameron Engh received his M.S. Computer Science from Brigham Young University in August 2008. Joseph L. Cooper received his M.S. in Computer Science from Brigham Young University in August, 2007. He is currently a doctoral student at the University of Texas at Austin. Julie A. Adams is an assistant professor in the Electrical Engineering and Computer Science Department at Vanderbilt University. She received her Ph.D. in Computer Information Sciences from the University of Pennsylvania in 1995.


UAVs in WiSAR 28

References Adams, J. A., Humphrey, C. M., Goodrich, M. A., Cooper, J. L., Morse, B. S., Engh, C., et al. (2008, To Appear). Cognitive task analysis for developing UAV wilderness search support. Journal of Cognitive Engineering and Decision Making. Alexander, A. L., Wickens, C. D., & Merwin, D. H. (2005). Perspective and coplanar cockpit displays of traffic information: Implications for maneuver choice, flight safety, and mental workload. International Journal of Aviation Psychology, 15(1), 1–21. Beard, R., Kingston, D., Quigley, M., Snyder, D., Christiansen, R., Johnson, W., et al. (2005). Autonomous vehicle technologies for small fixed-wing UAVs. Journal of Aerospace Computing, Information, and Communication, 2. Behavior characteristics from the search and rescue society of british colombia. (n.d.). Blinn, J. (1988). Where am I? What am I looking at? IEEE Computer Graphics and Applications, 8(4), 76–81. Bourgault, F., Furukawa, T., & Durrant-Whyte, H. F. (2003). Coordinated decentralized search for a lost target in a Bayesian world. In Proceedings of the 2003 ieee/rsj international conference on intelligent robots and systems. Casper, J., & Murphy, R. R. (2003). Human-robot interactions during the robot-assisted urban search and rescue response at the world trade center. IEEE Transactions on Systems, Man and Cybernetics, Part B, 33(3), 367-385. Cooke, N. J., Pringle, H., Pederson, H., & Connor, O. (Eds.). (2006). Human factors of remotely operated vehicles (Vol. 7). Elsevier. Cooper, J., & Goodrich, M. A. (2008). Towards combining UAV and sensor operator roles in UAV-enabled visual search. In Proceedings of the acm/ieee international conference on human-robot interaction.


UAVs in WiSAR 29

Endsley, M., Bolte, B., & Jones, D. (2003). Designing for situation awareness: An approach to user-centered design. London and New York: Taylor and Francis. Engh, C. (2008). A see-ability metric to improve mini unmanned aerial vehicle operator awareness using video georegistered to terrain models. Unpublished master’s thesis, Brigham Young University. Goodrich, M. A., Cooper, L., Adams, J. A., Humphrey, C., Zeeman, R., & Buss, B. G. (2007). Using a mini-UAV to support wilderness search and rescue: Practices for human-robot teaming. In Proceedings of the ieee international workshop on safety, security, and rescue robotics. Goodrich, M. A., Morse, B. S., Gerhardt, D., Cooper, J. L., Quigley, M., Adams, J. A., et al. (2008). Supporting wilderness search and rescue using a camera-equipped mini UAV. Journal of Field Robotics, 25(1-2), 89-110. Goodrich, M. A., Quigley, M., Adams, J. A., Morse, B. S., Cooper, J. L., Gerhardt, D., et al. (2007). Camera-equipped mini UAVs for wilderness search support: Tasks, autonomy, and interfaces (Tech. Rep. No. BYUHCMI TR 2007-1). Provo, Utah, USA: Brigham Young University. Heth, D. C., & Cornell, E. H. (1998). Characteristics of travel by persons lost in albertan wilderness areas. Journal of Environmental Psychology, 18, 223-235. Hill, S. G., & Bodt, B. (2007). A field experiment of autonomous mobility: Operator workload for one and two robots. In Proceedings of acm/ieee international conference on human-robot interaction (p. 169-176). Arlington, VA. John, M. S., & Cowen, M. B. (1999). Use of perspective view displays for operational tasks (Tech. Rep.). Pacific Sciences and Engineering Group Inc, San Diego CA. Kaupp, T., Douillard, B., Ramos, F., Makarenki, A., & Upcroft, B. (2007). Shared environment representation for a human-robot team performing information fusion. Journal of Field Robotics, 24(11-12), 911-942.


UAVs in WiSAR 30

Milgram, P., & Kishino, F. (1994). A taxonomy of mixed reality visual displays. IEICE Transactions on Information Systems: Special Issue on Networked Reality, E77-d(12), 1321-1329. Morse, B. S., Gerhardt, D., Engh, C., Goodrich, M. A., Rasmussen, N., Thornton, D., et al. (2008). Application and evaluation of spatiotemporal enhancement of live aerial video using temporally local mosaics. In Proceedings of the ieee computer society conference on computer vision and pattern recognition. Murphy, R. R., Steimle, E., Griffin, C., Cullins, C., Hall, M., & Pratt, K. (2008). Cooperative use of unmanned sea surface and micro aerial vehicles at Hurricane Wilma. Journal of Field Robotics, 25(3), 164-180. Nelson, D. R., Barber, D. B., McLain, T. W., & Beard, R. (2006). Vector field path following for small unmanned air vehicles. In Proceedings of the american control conference. Quigley, M., Goodrich, M. A., & Beard, R. W. (2004). Semi-autonomous human-UAV interfaces for fixed-wing mini-UAVs. In Proceedings of the international conference on intelligent robots and systems. Schmale, D. G., Dingus, B. R., & Reinholts, C. (2008). Development and application of an autonomous unmanned aerial vehicle for precise aerobiological sampling above agricultural fields. Journal of Field Robotics, 25(3), 133-147. Szeliski, R. (2006). Image alignment and stitching: A tutorial. Foundations and Trends in Computer Graphics and Vision, 2(1), 1–109. Tao, K. S., Tharp, G. K., Zhang, W., & Tai, A. T. (1999). A multi-agent operator interface for unmanned aerial vehicles. In Proceedings 18th digital avionics systems conference. Vicente, K. (1999). Cognitive work analysis: Toward safe, productive and healthy computer-based work. Mahwah, NJ: Lawrence Erlbaum Associates.


UAVs in WiSAR 31

Voyles, R., & Choset, H. (Eds.). (2008). Journal of field robotics: Special issue on search and rescue robots (Vol. 25) (No. 1-2). Wang, R. F., & Spelke, E. S. (2000). Updating egocentric representations in human navigation. Cognition, 77, 215-250. Wickens, C. D., Olmos, O., Chudy, A., & Davenport, C. (1997). Aviation display support for situation awareness (Tech. Rep.). University of Illinois Aviation Research Lab Technical Report. Wickens, C. D., & Prevett, T. T. (1995). Exploring the dimensions of egocentricity in aircraft navigation displays. Journal of Experimental Psychology: Applied, 1(2), 110 – 135. Wildes, R. P., Hirvonen, D. J., Hsu, S. C., Kumar, R., Lehman, W. B., Matei, B., et al. (2001). Video georegistration: algorithm and quantitative evaluation. In Proceedings of the eighth ieee international conference on computer vision (Vol. 2, pp. 343–350 vol.2). Wong, E.-M., Bourgault, F., & Furukawa, T. (2005). Multi-vehicle Bayesian search for multiple lost targets. In Proceedings of the 2005 ieee international conference on robotics and automation (p. 3169-3174).


UAVs in WiSAR 32

Author Note This work was partially supported by the National Science Foundation under grant number 0534736. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the sponsoring organization. Michael Goodrich may be contacted at email:[email protected]


UAVs in WiSAR 33

Footnotes 1 These complementing types of information can potentially be used in other ways to support search.

Examples include using UAVs to track searchers to ensure their safety or to facilitate training, to plan paths over difficult terrain, and to enforce the intent of the incident commander. 2 The title of the person managing the entire search is the “incident commander” or IC, but we will

sometimes use the term “mission manager” to indicate that the UAV technical search team might be part of a bigger search team and thus be managed by someone who is then managed by the IC. 3 It would have been ideal to have all participants be members of Utah County Search and Rescue, but

this was not logistically feasible. One reason for considering a consensus-based method for prioritizing technology developments was to minimize the biases and lack of experience in the participants. 4 The interface was used in a field trial subsequent to the writing of this paper. This field trial

confirmed that the interface was indeed potentially useful for practical UAV-enabled WiSAR.


UAVs in WiSAR 34

Technology HW

StabVid

Proc

HAG

ImEn

Gim

UI

Waypts

OffSrch

IntGUI

Importance

9

3

2

3

2

0

0

0

0

0

Priority

0

5

4

1

6

0

0

0

4

4

Overall

0

15

8

3

12

0

0

0

0

0

Table 1 The resulting overall development scores from combining the importance and priority rankings: Overall = Importance × Priority.


UAVs in WiSAR 35

Figure Captions Figure 1. The technology development cycle. Figure 2. A temporally local mosaic display. New frames (highlighted here in green) are aligned and placed over the displayed frames. As new frames move beyond the current display, the entire display scrolls and older frames are pushed off of the display. Figure 3. Four possible perspectives in the augmented virtuality user interface. Figure (a). Chase perspective plus carrot-and-camera control model. Figure (b). Track-up perspective. Figure (c). North-up perspective. Figure (d). Pilot perspective. Figure 4. Instantaneous See-ability Figure 5. Rendering the Hiker Figure (a). A frame with no hiker. Figure (b). A frame displaying the hiker. Figure 6. Comparison of See-ability Techniques Figure (a). The flight path. Figure (b). Simple coverage map. Figure (c). See-ability: Unique Angles.


UAVs in WiSAR 36

Figure (d). See-ability: Cumulative.


Field Trials

Flight Tests

Technology Development

ecological validity

frequency of use

User Experiments



(a) Chase perspective plus carrot-and-camera

(b) Track-up perspective.

control model.

(c) North-up perspective.

(d) Pilot perspective.



(a) A frame with no hiker.

(b) A frame displaying the hiker.


(a) The flight path.

(b) Simple coverage map.

(c) See-ability: Unique Angles.

(d) See-ability: Cumulative.