Visualization Viewpoints Editors: Theresa-Marie Rhyne and Lloyd Treinish
Real-Time Visualization in the Offshore Industry ______ Paul Chapman and Peter Stevens Sonar Research and Development Derek Wills and Graham Brookes University of Hull
R
eal-time underwater visualization has been extremely slow to develop within the offshore industry and has generally been limited to 2D representations of vessel positions on digital charts.1 Only recently have marine industries realized the potential of 3D real-time virtual environments (VEs) and visualization systems for effective management planning and real-time situation awareness.2,3 In this article, we describe a real-time visualization of the clean-up of a former US Naval Submarine Base, located in Holy Loch, Scotland. (See the sidebar “Offshore Diamond Mining” for another implementation.) Our Whole-Field Modeling System (WFM) has provided an accurate real-time visualization of numerous varying parameters such as remotely operated vehicles (ROVs), cranes, barges, grabs, magnets, and detailed seabed topography. The system improved the field staffs’ spatial and temporal awareness and facilitated decision making within the complex offshore working environment.
The clean-up Holy Loch lies approximately 50 km due west of Glasgow and is 4 km long and 1 km wide. Britain and the US have used the loch for military purposes since World War I. In 1945, the US military returned to the loch to act as peace keepers for the North Atlantic Treaty Organization alliance. The US Navy operated a submarine base in the loch for more than 30 years with more
than 1,500 American servicemen stationed there. In 1992, the US closed the base, and the American military withdrew, leaving behind much military debris. Before the base can be returned to the Clyde Port Authority, the loch must be carefully cleared by the UK of all debris. We initially surveyed Holy Loch in 1996 using our sonar Seabed Visualization System (SVS).4 The resulting high-resolution bathymetric data geographically highlighted the main debris areas, which were mostly located under the original floating docks. Figure 1 shows a depth-colored bathymetric plot from the sonar survey showing the seabed that lies directly under the original floating naval base. The original heavy anchor chains that secured the docks caused the scour marks in the loch floor. The US military originally found the deep circular hole in the image’s center to be one of the deepest locations in the loch and consequently used it as the floating docks’ main position. The clean-up team split the area for debris clearance in Holy Loch into several 25-m2 blocks. They broke down each block into four 12.5-m2 areas. These blocks consist of 9 × 9 individual lifting zones, each approximately 1.3 m2 (the same footprint as a lifting grab). The clean-up crew maneuvered a large clearance barge via four anchor wires running from each corner of the vessel to secure anchors on the loch floor. By slackening or tightening the anchor wires, they positioned the barge to within meter accuracy over a target area. The barge contained a lifting crane and control cabins. Figure 2 gives a simplified model of the clearance operation. Initially, we presurveyed a working block using the Begin
1
Bathymetric survey of the former US naval base. Area shown is approximately 300 m2.
Presurvey new block with SVS Block pass ROV video survey
Grab and magnet drop zones Block fail Postsurvey block with SVS
2 A simplified model of the clearance operation’s survey procedure. 6
July/August 2001
0272-1716/01/$10.00 © 2001 IEEE
Offshore Diamond Mining We’ve also extended our real-time WFM visualization system to facilitate operations within the offshore diamond mining industry in South Africa. Namco implemented WFM onboard their diamond mining vessel MV Kovambo, off the coast of Namibia between August and September 2000. A specially trained pilot remotely controlled a 180-ton crawler that tracked across the ocean floor mining for diamonds. The crawler used a cutting head to scour and break the diamond-bearing sediment into small spoil that was pumped via a 1-m-wide umbilical back to the mother ship. Once on the Kovambo, the cuttings were screened for diamonds and the spoil deposited back onto the seabed. The objective of this mining visualization project A Real-time diamond mining. The green area indicates was to provide the tool pilot and party chief with a the unmined seabed. real-time display of the tool’s position and orientation, including an accurate visualization of the tracks, torso, boom, and dredge-tip. We also designed the system to provide the client with a true real-time visualization of the seabed topography surrounding the crawler. The seabed terrain data was continuously updated from individual ping returns direct from the sonar transducers mounted on the crawler. Figure A shows the display provided to the crawler’s pilot. In this example, the unmined (green) seabed is visible to the pilot. The pilot tracked the tool using the same pinger and motion sensor systems we used for the ROV at Holy Loch. The crawler properties, such as boom and knuckle angle, were captured from an information stream already output from the crawler system. Noise from the sonar transducers caused the small stalagmites visible in Figure A. They were corrected (removed) as the sonar system swept back over the affected area. It’s important to remember that this is a harsh underwater environment caused by pneumatic noise and silt and debris floating around the sonar transducers. The WFM diamond mining implementation has facilitated operations by ■
■ ■
reducing tool “stickages” generally caused by digging a hole too deep and consequently preventing the pilot from being able to track out of the hole. We prevented this by giving the pilots an accurate visualization of the tool’s surrounding seabed. monitoring deployment and recovery procedures. A presurvey and online deployment monitoring prevented lowering the tool into problematic areas. mining to consistent depths. Using the depth-corrected color bathymetric data allows systematic coverage of the prospect area.
We briefly considered implementing stereo displays—for example, using liquid crystal shutter glasses— but rejected this because of the cramped and hazardous nature of offshore working environments.
SVS subsurface transducer array. This survey identified the exact location of any debris within the 81 lifting zones. After the sonar presurvey, the crew positioned the barge for grabbing and magneting. Initially, the barge crew fitted the grab to the crane’s block and lifting began. A crane driver lowered the grab to the seabed, closed it, and raised it 3 m off the loch floor. If the crane’s load-cell indicated that there was debris in the grab, then the operator raised the debris to the surface. The crane driver continually lowered the grab to the seabed over a particular drop zone until no more debris was recovered. All debris collected from the main crane was carefully lowered onto the wash-down deck and immediately inspected. The crew took extreme care when
lowering debris onto the wash-down deck. Bomb disposal experts detonated any suspicious or potentially explosive material away from the salvage barge. The wash-down crew then cleaned the silt and mud off the debris using two high-powered water cannons. An Environmental Agency representative subsequently checked the debris using a Geiger counter for any abnormal radiation levels to ensure that the gamma radiation didn’t surpass the background level. The crew then broke the debris into sections using hydraulic cutters and lifted it into skips (large metal storage boxes for rubbish) using a secondary grab. A computer database kept accurate records of the lift storing information such as wet weight, dry weight, and debris classification. The
IEEE Computer Graphics and Applications
7
Visualization Viewpoints
3 Real-time debris clearance. (a) GPS receiver used to calculate the crane’s position. (b) Sonar responder that helps position the underwater crane block. (c) Operation control cabins.
the grab and magnet, and the collection of bathymetric terrain data collected from the sonar surveys. In reality, the clearance procedure wasn’t as linear as the model in Figure 2. For example, while the crane grabbed debris from a particular block, the ROV surveyed a previously grabbed block. Idle operations can be extremely expensive offshore, so effective time and resource management are essential. With the existing 2D display system, it was difficult to conceptualize quickly the location of moving objects in relation to other objects. For example, it was imperative that the ROV pilot know his vessel’s position in relation to the electromagnet under the water. A collision between the ROV and a 5-ton grab or magnet could be extremely costly. Our objective was to provide users with a real-time visualization of the clean-up operation to facilitate spatial and temporal awareness of all the components used within the clean-up operation.
Real-time implementation 4 Grab breaking water with debris flag set.
skips were finally placed into an adjacent barge for later removal from the site and recycling. When a drop zone was clear, the crane driver moved the grab to the next adjacent drop zone. After clearing all 81 zones, the crew replaced the grab with a large electromagnet and repeated the clearance process for all 81 zones. The magnet collected smaller metallic debris from the loch bed that might have slipped through the grab—for example, gas bottles and shells. After magneting, we performed a postsurvey of the area using the barge-mounted transducers. We then deployed a Phantom Mark II ROV from the back of the barge to visually inspect the block. The ROV flew over the block in nine runs providing video data of the entire area. We knew the ROV’s precise position at all times so we could geographically tag any debris it located. After the ROV finished the video survey, the block either passed or failed inspection. A failed block needed to be remagneted or regrabbed (depending on the nature of the debris remaining) in the problem areas identified by the ROV video survey. If the block passed, the crew moved the barge to the next block and repeated the entire process—presurvey, grab and magnet, postsurvey, and ROV survey.
Complexity of operations Clearly, there was a lot happening during this process: the ROV flying over the seabed, the barge’s movement, the crane arm’s movement, the position of
8
July/August 2001
Figure 3 shows a snapshot from WFM’s real-time visualization of the clean-up operation. The image depicts the barge in its actual position including a correctly modeled and positioned crane preparing to drop some debris onto the wash-down deck. We created the 3D models in the visualization using 3D Studio Max and detailed them with engineering drawings. We then imported the models into WFM and modified them for functionality and real-time manipulation. We included rolling clouds and mountains to make the VE as realistic as possible. We calculated the barge’s location using real-time kinematic (RTK) satellite tracking that provides centimeter positional accuracy. Motion sensors provide rotational information: heave, pitch, and roll. We took yaw (heading) from the vessel’s compass. We also knew the main crane’s physical dimensions including its offset from the center of the vessel. A GPS receiver on the tip of the crane jib (see Figure 3a) let us calculate the crane’s heading and pitch. We calculated the underwater positioning of the grab and magnet by placing a responder onto the crane block (see Figure 3b). As the crane block traveled toward the loch floor, the responder sent a signal at a known frequency. Two transducer arrays listened in two orthogonal planes for the responder’s return signal. The SVS then provided a bearing and range for each of the planes letting us calculate the responder’s exact 3D position to centimeter accuracy. A load-cell on the crane winch provided a metric for lift data in kilograms. This weight was “zeroed” each time the crew attached the grab or magnet to the crane block, improving the accuracy of the lifting sessions. As soon as the weight on the load-cell exceeded a predefined threshold, a debris flag was set and WFM rendered debris under the grab or magnet (see Figure 4). The exact nature of the debris wasn’t known, so an exact visualization was impossible because we only knew the object’s weight. However, to give some indication of the weight from the load-cell, we scaled the debris’ volume proportionately to the amount being lifted (assuming homogenous debris). The large grab had five hydraulic arms and stood
approximately 2-m tall with a span of 1.5 m2. The grab could be opened, closed, or moved between states. We measured the pressure applied to the grab to open and close the arms using pressure sensors and then sent it to WFM. Consequently, we were able to animate the opening and closing of the grab in real-time. The 5 ton electromagnet had a 1.5 m diameter and could be either on or off. The amount of current passing through the magnet was sent to WFM so that it could be colored according to its state: yellow for off and red for active or live. The system converted the bathymetric data read from the SVS sonar transducers fixed on the barge into digital terrain maps (DTMs) in real time and imported it automatically into the WFM model. We color-textured the terrain according to depth. To provide rendering efficiency, WFM used multiple levels of detail for the terrain depending on the viewer’s position. We created simplified data sets by undersampling the high-resolution DTMs. We modified the Phantom ROV to include motion sensors, responders, and a specially designed miniature SVS developed at SRD. We calculated the ROV’s position in the same way as the crane’s block and detected it using a secondary SVS barge mounted transducer system. All external data such as bathymetry, transponders, and crane positions were streamed into the WFM visualization over a 100 Base-T network. We implemented WFM on a Pentium III 600-MHz PC using nVidia graphics technology and developed it using Microsoft’s Visual C++ version 6 and OpenGL graphics libraries.
Problems encountered We developed the majority of this visualization system at Sonar Research and Development’s headquarters in Beverley, England, by simulating operations from captured data recorded in Holy Loch. Actual implementation of the system on the barge revealed several unforeseen problems. We originally flagged debris based on the value returned from the crane’s load-cell. However, the magnet had a greater surface area than the grab and consequently registered significantly more drag underwater. As a result, the crane required more energy to pull the magnet through the water column than the grab. We therefore needed to set different load-cell debris thresholds for the grab and the magnet. One small obstacle we had was that we couldn’t track the crane block’s responder in less than 2 m of water due to the nature of the sonar equipment. Consequently, when the crane driver lifted the grab or magnet out of the water, we lost the crane’s block depth positioning. We therefore introduced a rule that if WFM hadn’t heard from the block’s responder within the last four seconds, then the block must be out of the water. We then rendered the block at a fixed height of 6 m above the water level. The crane’s heading and pitch wasn’t affected by the inability to track the block out of the water because the crane’s positioning relies on the GPS receiver placed on the tip of the crane jib. As a result, we always had the crane’s correct position so the system can visualize the crane driver depositing the debris on the wash-down deck.
5 Under the barge. WFM provides a global picture of clean-up operations. The user can pilot a virtual vehicle around the VE to select areas of interest.
Results WFM rendered to several displays within the control cabins on the barge (see Figure 3c): specifically, the ROV, crane, control, and party chief cabins. We multiplexed the displays so viewers could switch between various predefined views (such as under the barge, ROV camera viewpoint, and fixed crane grab cameras). No training was required for WFM because we didn’t need to teach the staff how to read and interpret the natural viewing environment. We tested the visualization system on five different ROV pilots and control staff, two crane drivers, and two party chiefs. One problem for the crane drivers is that their work can become monotonous, increasing the potential for error. A typical mistake would be forgetting to sensitize the magnet on the loch floor. Supplying the crane driver with the WFM 3D view (rendering the state-dependent colored magnet) eradicated this error completely. The crane drivers also reported that they felt more comfortable knowing the whereabouts of the ROV underwater. Control cabin staff no longer needed to shout to the adjacent ROV cabin to confirm that the ROV was in a safe position prior to a barge move because they knew its exact position in relation to the barge. The cabin staff reported that WFM provided a refreshing change from the numerous graphs and sound signals emanating from the PCs within the control cabin. We provided the ROV pilots with a WFM 3D display that augmented their 2D ROV displays. On occasion, the nearby grab lifting operations would disturb the silt on the loch floor. This sediment upheaval reduced the ROV camera visibility to zero. The WFM system was unaffected by poor visibility and continuously provided the pilots with a crystal-clear visualization of their ROV underwater (see Figure 5). The main benefit of WFM reported by the party chief (responsible for all operations offshore) was WFM’s ability to provide a more complete picture of the clean-up operation in real time and in a format that the staff could immediately comprehend. The party chief also had the added luxury of a force-feedback joystick to maneuver a virtual vessel to select any viewpoint or area of interest within the real-time model. This included the ability to closely examine anomalies on the loch floor generated by the sonar data. All parties welcomed the visualization and, although
IEEE Computer Graphics and Applications
9
Visualization Viewpoints
we didn’t design the system to replace the current software tools used for the survey, it augmented, enhanced, and improved operations. The primary benefit reported from WFM’s visualization was a significant improvement in spatial and temporal perception of all components used in the survey. WFM’s real-time implementation improved the clean-up operation’s effectiveness and provided the field staff with an improved understanding of the underwater environment. This reduced the potential for error, which in the offshore industry can be extremely costly, both financially and ecologically. Readers can download papers, images, and animations related to this work from http://www.hull.ac. uk/php/csspmc. ■
Acknowledgments We thank the British Ministry of Defense, Namco, and all the staff at Sonar Research and Development. Special thanks go to Rafe Montgomery, Kevin Brown, Mark Thomas, James Ward, and Helen Wright. The Teaching Company Directorate partially funds this work.
References 1. G. Hatcher and N. Maher, “Real-Time GIS for Marine Applications,” Marine and Coastal Geographical Information Systems, D. Wright and D. Bartlett, eds., Taylor & Francis, New York, 2000, pp. 137-147. 2. P. Chapman et al., “Real-Time Visualization of the ClearUp of a Former US Naval Base,” Proc. Visualization 2000, ACM Press, New York, 2000, pp. 505-508. 3. K. Belcher and C. Moore, “3D Graphics Technology Changes Deepwater Installation Practices,” Sea Technology, vol. 42, no. 1, Jan. 2001, pp 49-51. 4. P. Chapman et al., “Seabed Visualization,” Proc. Visualization 98, ACM Press, New York, 1998, pp 479-481.
Readers may contact Chapman at the Dept. of Computer Science, Univ. of Hull, Cottingham Rd., Hull, UK, email
[email protected]. Readers may contact department editors Rhyne and Treinish by email at
[email protected] or lloydt@ us.ibm.com.
From the publisher of Real-Time Rendering
Crystal-Clear for the Serious Programmer… Realistic Ray-Tracing Peter Shirley “I enjoyed this small, elegant book on how to write a modern tracer. As someone who has written his share of rendering programs, it’s always a pleasure for me to see someone capture the essence of the idea as well as Shirley has done here…If you’re ready to roll up your sleeves and implement a state-of-the-art ray-tracer, and are comfortable with basic calculus, this book will serve you as an ideal roadmap and reference.” —amazon.com Hardcover, 184 pp. $35.00
10
July/August 2001
Realistic Image Synthesis Using Photon Mapping Henrik Wann Jensen New at SIGGRAPH 2001
This book is a practical guide to photon mapping: it provides both the theory and the practical insight necessary to implement photon mapping an simulate all types of direct and indirect illumination efficiently. Hardcover, 200 pp. $39.00
A K Peters, Ltd. 63 South Avenue, Natick, MA 01760-4626 Tel: 508 655-9933 Fax: 508 655-5847
