THE HIGH RESOLUTION STEREO CAMERA (HRSC) – DIGITAL 3D-IMAGE ACQUISITION, PHOTOGRAMMETRIC PROCESSING AND DATA EVALUATION F. Scholten, S. Sujew, F. Wewel, J. Flohrer, R. Jaumann, F. Lehmann, R. Pischel, G. Neukum German Aerospace Center (DLR) Institute of Space Sensor Technology and Planetary Exploration Rutherfordstr. 2 D – 12489 Berlin, Germany email:
[email protected] phone: ++49-30-67055-326 fax: ++49-30-67055-402 ABSTRACT Digital techniques replaced more and more the analogue photogrammetric analysis in daily work during the past years. The consequent way to complete the digital line is to perform digital data acquisition. Since digital frame cameras for photogrammetric purposes will not be available within the next years, digital line scanners can fill this gap. At the Institute of Planetary Exploration of the German Aerospace Center (DLR) the High Resolution Stereo Camera (HRSC) has been designed for international missions to planet Mars. During the past two years an airborne version of this camera, the HRSC-A, has been successfully applied in many flight campaigns and in a variety of different applications. The HRSC-A fulfils all requirements of a photogrammetric sensor. It is based on the along-track triple-stereo principle even using 9 CCD arrays and combines 3D-capabilities and high resolution with multispectral data acquisition. Variable resolutions depending on the camera control settings can be generated. A high-end GPS/INS system in combination with the multi-angle image information yields precise and high-frequent orientation data for the acquired image lines. In order to handle these data a completely automated photogrammetric processing system has been developed in cooperation between the Department for Photogrammetry and Cartography of the Technical University of Berlin and the DLR. This system is capable to generate impressive multispectral 3D-image products of the HRSC-A data combined with accuracies in planimetry and height of better than 0.1 thousandth of the flight altitude, accuracies which have been confirmed by detailed investigations.
1. INTRODUCTION During the past 20 years practical photogrammetry has undergone fundamental changes from analogue to analytical and finally to digital processing techniques. Nevertheless, image acquisition is still dominated by analogue systems because of the extremly high demands on the digital imaging systems. High data rates and the permanent camera movements have to be handled. At the DLR-Institute of Space Sensor Technology and Planetary Exploration the HRSC was developed as a multispectral multi-line/multi-stereo scanner system for the exploration of Mars (Neukum & Tarnopolsky, 1990; Albertz et al., 1992; Neukum et al., 1995) and will be flown onboard the European Mars Express mission in 2003 (Neukum et al., 1999). A modified version has been established for airborne
applications, the HRSC-A. It fulfils all radiometric and geometric requirements of a photogrammetric camera system and is equipped with a completely automated photogrammetric processing system, yielding high-accurate images and 3D-products (Wewel et al., 1998). The special properties of the HRSC-A and the photogrammetric products have been used during many applications of different types. 2. DATA ACQUISITION WITH THE HRSC-A The High Resolution Stereo Camera (HRSC) is a multiple line pushbroom instrument. Nine superimposed image tracks are acquired simultaneously (along-track) by nine CCD line sensors mounted in parallel and behind one single optics (see Figure 1).
platform (ZEISS T-AS) in order to damp mechanical vibrations and to enforce near-nadir viewing. Position and orientation during flight navigation are measured continuously by means of differential GPS and INS. HRSC-A/QM Technical Parameters Focal Length:
175 mm
Total Field of View:
37.8° x 11.8°
Number of CCD Lines: 9 Figure 1. Imaging Principle of the HRSC-A Five of these are panchromatic sensors arranged at specific viewing angles and provide the multiplestereo and photometric capabilities of the instrument. Four of the nine CCD lines are covered with different filters for the acquisition of multispectral images. According to its development for space missions, the camera has small dimensions, low mass, low power consumption and a robust design. A slightly modified version of the instrument has been adapted for operation in terrestrial airborne remote sensing. To optimize image radiometry, the recording levels of the individual channels are controlled separately by adjustable gain factors.
Figure 2. The HRSC-A The HRSC-A (see Figure 1 and Table 1) is identical in its main structural features to the original HRSC system and includes its original optics. Some additional electronics have been added to meet specific airborne requirements. During image acquisition, data rates of 10 MByte/s provided by four parallel signal chains can be stored on a high-speed tape recorder. The camera is mounted on a stabilized
Stereo Angles:
±18.9° and ±12.8°
Pixels per CCD Line:
5184 (active)
Pixel Size:
7 µm
Radiometric Resolution: Scan Rate:
10 bit reduced to 8 bit
Mass:
camera 12 kg (32 kg incl. subsystems)
max. 450 lines/s
Table 1. HRSC-A Technical Data 3. GPS/INS DATA PROCESSING For the position and attitude determination of the HRSC-A camera the integrated GPS/INS system APPLANIX POS/DG (Hutton & Lithopoulos, 1998) was used. The Position and Orientation System (POS) consists of two components, the self-contained and separeted Inertial Measurement Unit (IMU) and the POS Computer System (PCS). A high-performance Litton LR-86 IMU was used. It contains Litton A4 navigation grade accelerometers and Litton G7 drytuned gyros (DTG). The IMU was directly mounted on the top of a HRSC-A frame for precise measurements of camera motions. It provides data of incremental velocities and angular rates with an output data rate of 200 Hz. The POS also houses a GPS receiver L1/L2 Novatel Millenium Card. The GPS antenna was mounted directly above the HRSC-A camera. The POS provides logging of raw IMU data and of raw GPS data to the 8 mm tape of the PCS for further use in GPS/INS post-processing.
The first step of post-processing is to combine raw GPS data from POS GPS receiver and from the ground reference station for a kinematic GPS solution. When the ground station was installed directly in the surveying area, the integer carried phase GPS trajectory ) is achieved with a position accuracy of 1 cm for East, North and of 2 cm for height. The second step of post-processing is the optimal integration of GPS and inertial data which was taken with APPLANIX software POSProc. The POSProc realises an aiding inertial navigation algorithm, which combines the inertial data from IMU with the GPS position and velocity to calculate high accurate position and attitude, called best estimate of the
trajectory (BET). Since the typical errors of inertial navigation and GPS measurements are complementary, the used Kalman filter utilization makes it possible to calibrate inertial and GPS errors against each other. The resulting position accuracy is determined principally by the GPS solution and the attitude accuracy depends on the quality of IMU inertial sensors. After the GPS/Inertial processing the HRSC-A camera position is available with accuracies of ±2-3 cm, while the angular accuracy of the sensor orientation is ±0,004° (roll and pitch) and ±0,008° (heading) with a data rate of 200 Hz.
Figure 3. Photogrammetric Processing Line for HRSC-A Data 4. DIGITAL PHOTOGRAMMETRIC DATA PROCESSING The digital photogrammetric processing system was developed basically at the Technical University of Berlin for the Mars96 mission (Scholten, 1992; Uebbing, 1992; Wewel, 1992). A completely automated procedural software system has been built up for airborne application (Figure 3). It makes use of a set of systematically preprocessed image, orientation and calibration data. In a first step, the absolute orientation of each image line is computed using the post-processed GPS/INS data. With the help of identical points derived from the different stereo channels and from overlapping image strips the angular offsets between the INS and camera axes can be estimated in an optimization procedure.
In the optimal flight mode with parallel contrary image strips no additional ground control point information is needed. Additional perpendicular strips stabilize the estimation. In early flight campaigns a conventional INSsystem has been used. For this constellation a comprehensive procedure was developed modelling the INS-camera offsets as well as the gyro offsets and drift parameters with high accuracy. Based on the calibration data and applying the INS/camera offsets to the orientation data each image line is projected to an artifical surface which is defined by the mean elevation of the imaged region. Figure 4 shows the potential of the rectification process an the quality of the GPS/INS
data, even under the extreme conditions during a flight maneuver. The flight movement corrected data set already defines a first product level which
can be used for interpretation purposes, but it still contains dislocations due to the shape of the terrain.
Figure 4. Effectiveness of Geometric Correction under Extreme Conditions Original HRSC-A Image Data (lower left), Pre-Rectified Image Data (center and upper right) Flight Altitude 3000 m, Ground Resolution 15 cm One fundamental quality of the HRSC-A is, beside its high resolution and multispectral capabilities, its multi-stereo functionality. Based on the pre-rectified image data a multi-image matching technique is applied to derive conjugate points in each of the five stereo channels. The algorithm makes use of areabased correlation in image pyramids to derive approximate values, which are refined with subpixel accuracy by least-squares matching. Due to the five different viewing angles, the complete image area can be matched even in urban areas (where gaps are more likely to appear when systems providing only two stereo observations are used). The image coordinates of the pre-rectified images are finally back-transformed to original image coordinates. Together with the calibration data and the orientation information each set of image coordinates define a bundle of rays in object space. The final object point is the intersection of these rays and is derived by least-squares adjustment. Since there are up to five rays defining an object point, it is possible to estimate point accuracies and to use these estimates for the detection and elimination of blunders, which is essential for the generation of high-quality 3D-products.
A Digital Elevation Model (DEM) can now be calculated from the set of object points. Since these points are still defined in the GPS domain (the WGS84 system), they have to be transformed by means of a datum shift to any reference system which is used as basis for the requested map projection. The final raster DEM is generated using different interpolation techniques depending on the surface type. Possible gaps (due to matching failures or rejected object points) are filled using pyramidal interpolation. The DEM, now given in any requested map projection, defines the next type of product. It can be used for the extraction of profiles, contour lines or other DEM follow-up products (see Figure 5). Finally, it is the basic prerequisite for the following generation of orthoimages.
5. GEOMETRIC VALIDATION OF THE HRSC-A CAMERA SYSTEM A thorough geometric validation of the HRSC-A system has been performed over different test targets. The DLR test area, an open coal mine close to Cologne/Bedburg, was measured in parallel by means of ground based GPS and aerotriangulation using images acquired by a conventional aerial surveying camera. 29 of 120 signalized ground control points were measured by GPS. The remaining points were determined by aerotriangulation with a standard deviation of 10 cm.
Figure 5. Pre-Rectified HRSC-A Image of a Hillside and Perspective View of the DEM Grid (Flight Altitude 3000 m, Ground Resolution 15 cm) In the generation of orthoimages, the rays defined by the calibration and the orientation data are intersected with the surface described by the DEM. To reduce computation time, this is done only for a subset of all pixels, the so called anchor points. They define a regular grid in the image, dense enough to describe orientation and DEM variations. Based on the intersections of these points all pixel positions within the anchor point patches are backtransformed into the original image and the greyvalues for the final orthoimage can be interpolated. The final step within the photogrammetric data processing is the generation of orthoimage mosaics using orthoimages of adjacent strips. Multiple information within the overlapping regions is used to derive correction tables for the histograms of each image strip. The result is a homogeneous image mosaic for each spectral band (see Figures 6 and 8). The color bands can be combined with the generally higher resolution of the panchromatic nadir channel by means of IHS color transformation, thus yielding color mosaics including the high geometric resolution of the panchromatic data.
Three image strips with 50 % side-overlap and two addional strips across the track were flown within one hour. The HRSC-A was operated onboard a Cessna 208 from an altitude of 3,000 m and with a read-out frequency of 450 lines per second. The corresponding approximate ground pixel resolutions are 15 cm for the nadir channel and about 30 cm for the remaining panchromatic stereo and photometry channels. The position and attitude of the inertial measurement unit (IMU) was determined by the APPLANIX post processing software (see chapter 3). The exterior orientation data were estimated with an accuracy of ±2,2 cm for position, ±0,0004° for roll and pitch angle and ±0,008° for heading. Four ground control points and about 100 image tiepoints have been used to determine the installation offsets between the HRSC camera and the IMU axes. According to the mean intersection accuracy, a mean relative point accuracy of ±10-12 cm at 6 million DEM points has been achieved. The absolute accuracy of the check points of each image strip is given in Table 2. Strip 1 2 3 4 5 1-5
No. of Points 44 51 36 6 10 120
Horizont. Acc. 12 11 16 18 14 13
Vertical Acc. 19 21 16 24 9 18
Point Acc. 22 24 23 30 17 23
Table 2. HRSC-A Control Point Accuracies in cm
The mean absolute accuracy determined from the 120 independent check points with respect to the independent point measurements is ±23 cm, (mean vertical and horizontal accuracies of ±18 cm and ±13 cm). These results have been achieved based on directly measured exterior orientation data without any improvement through a bundle adjustment. 6. APPLICATION EXAMPLES Since the first airborne experiments (May 1997), the HRSC-A system has been used for many different applications (Lehmann et al., 1998). Simultaneous high resolution multispectral orthoimages and DEM data have been acquired for applications as different as volcano monitoring, mapping of urban areas, open coal mines, flood hazards, and of coastal zones. The derivation of morphological characteristics such as slopes, volumes, or drainage patterns through the analysis of DEMs is an essential step for many geoscientific and environmental applications. DEM analysis frequently is combined with the analysis of remote sensing imagery, for example to classify and measure topographic changes related to volcanic activity, landslides or avalanches. Planning requirements for urban areas increasingly involve GIS technology, e.g. using 3D models for the needs of the mobile phone service in the telecommunication industry. The need for both high resolution multispectral images and DEM data can be addressed by airborne digital imaging with the HRSC-A. 6.1 Applications in Volcanology Figure 7. Orthoimage Mosaic of Vulcano Island based on 7 HRSC-A Image strips In May 1997, a flight campaign at the Aeolian Islands (Italy) was carried out to assess the potential of this new data source for applications in volcanology and comparative planetology (Gwinner et al., 1999). Vulcano Island is located 30 km north of Sicily and covers an area of 21,2 km2. The island is of complete volcanic origin and bears the active volcano Fossa di Vulcano (391 m).
Figure 6. Detail of Orthoimage Mosaic of Vulcano (see Fig. 7) with Contour Lines (equidistance 10 m).
The HRSC-A was operated onboard a Dassault Falcon DA 20 from an altitude of 5,000 m, covering the entire island with 7 image tracks. The corresponding approximate ground pixel sizes are 25 cm for the nadir channel, 50 cm for the remaining panchromatic stereo and photometry channels, and 100 cm for the multispectral channels.
A DEM on a 160 cm grid and a multispectral orthoimage mosaic have been derived for the whole island (Figures 7 and 8), demonstrating the suitabi-
lity of the system for target areas with high topographic relief and weakly textured surfaces (Figure 6).
Figure 8. Shaded Perspective View of HRSC-A DEM of Vulcano Island 6.2 Mapping of Urban Areas The potential of the the HRSC-A system for photogrammetric surveys in urban areas has been shown in its operational use at several European cities. Specifically, the availability of five stereo observations, including the near-nadir looking photometry channels, is beneficial for the measurement of man-made objects typically including steep surface discontinuities.
Figure 11 shows the progress of the reconstruction of the German parliament in 1998 and 1999. Figure 12 is a greyvalue-coded visualization of the central Berlin DEM. The dark patches arranged in north-south oriented bands are open construction sites for subground infrastructure.
Fig. 9-12 show results of two campaigns at Berlin, Gernany carried out in 1998 and 1999.
Figure 13 shows an example (Lisbon, Portugal) of a joint ISTAR/DLR project on European cities for HRSC-A application in the telecommunication market (Renouard & Lehmann, 1999).
The orthoimage mosaic (Figure 9) is based on seven image tracks acquired from a flight altitude of 3000 m onboard a Cessna 208, and with a maximal ground resolution of 15 cm. Figure 10 is a series of different steps of magnification of Figure 8, showing the HRSC-A capabilities imaging large areas in high resolution.
The cartographic potential of the HRSC-A system was demonstrated by the generation of a map sheet of the Image Map 1:5,000 series of the city of Berlin in cooperation with the "Senatsverwaltung für Bauen, Wohnen und Verkehr, Berlin". Due to the high resolution, the image data can also be used for large scale applications in scales of up to 1:500.
Figure 9. Orthoimage Mosaic of Central Berlin, 1998.
Figure 10. Different Magnifications of the Upper Right Part of Figure 8, Berlin, 1998.
Figure 11. Reconstruction of the German Parliament Building in 1998 (left) and 1999 (right)
Figure 12. DEM of Central Berlin, 1998
Figure 13. Portion of a Shaded DEM of Lisbon (Portugal), processed by ISTAR (France) using HRSC-A Data
7. CONCLUSIONS The HRSC-A has demonstrated its unique potential in modern digital image acquisition within several flight campaigns. The achieved accuracies show that the combination of a multispectral digital line scanner with a comprehensive processing system
yields high accurate data products even under difficult conditions and for large regions. More than this, the digital techniques enables new fields of geoscientific, environmental and cartographic applications while minimizing analogue to digital data transition and cost intensive manual interactions.
ACKNOWLEDGEMENTS We would like to acknowledge the scientific and operational contributions of all persons of the DLR, which are involved in the HRSC-A developments and flight campaigns. REFERENCES J. Albertz, F. Scholten, H. Ebner, C. Heipke, G. Neukum, “The Camera Experiments HRSC and WAOSS on the Mars94 Mission“. International Archives of Photogrammetry and Remote Sensing, Vol. 29, Part B1, pp. 216-227, 1992. K. Gwinner, E. Hauber, H. Hoffmann, F. Scholten, R. Jaumann, G. Neukum, M. Coltelli, G. Puglisi, “The HRSC-A Experiment on High Resolution Imaging and DEM Generation at the Aeolian Islands“. 13th Int. Conf. Applied Geologic Remote Sensing, Vancouver, Canada, Vol. I, 560-569, 1999. J. Hutton, E. Lithopoulos, Airborne Photogrammetry Using Direct Camera Orientation Measurements“. Photogrammetrie-FernerkundungGeoinformation, 6, E. Schweizerbart’sche Verlagsbuchhandlung, Stuttgart, Germany, pp. 363-370, 1998. F. Lehmann, M. Brand, T. Bucher, S. Hese, A. Hoffmann, S. Mayer, F. Oschütz, S. Sujew, T. Roatsch, F. Wewel, Y. Zhang, "Data Fusion of HYMAP Hyperspectral and HRSC Multispectral Stereo Data: Remote Sensing Data Validation and Application in Different Disciplines". 1st Earsel Workshop on Imaging Spectroscopy, Zurich, Switzerland, pp. 105-119, 1998. G. Neukum and V. Tarnopolsky, "Planetary Mapping – The Mars Cartographic Data Base and a Cooperative Camera Project for 1994". GeoInformationssysteme, 3, H. 2, pp. 20-29, 1990. G. Neukum., J. Oberst, G. Schwarz, J. Flohrer, I. Sebastian, R. Jaumann, H. Hoffmann, U. Carsenty, K. Eichentopf, R. Pischel, “The Multiple Line Scanner Camera Experiment for the Russian Mars96 Mission: Status Report and Prospect for the Future“. In Photogrammetric Week ‘95, eds. D. Fritsch & D. Hobbie, Wichmann, Heidelberg, Germany, pp. 4561, 1995.
G. Neukum and the HRSC Co-Investigator and Experiment Team, "The High Resolution Stereo Camera (HRSC) onboard the European Mars Express Mission". ESA, 12 p., 1999 (in press). L. Renouard, F. Lehmann, "Digital Aerial Survey Data for Telecoms Network Planning: Practical Experience with a High-Resolution Three-View Stereo Camera". OEEPE Workshop on Automation in Digital Photogrammetric Production, Paris, France, 21-24 June 1999. F. Scholten, “Automated Generation of Coloured Orthoimages and Image Mosaics Using HRSC and WAOSS Image Data of the Mars96 Mission“. International Archives of Photogrammetry and Remote Sensing, Vol. 31, Part B2, pp. 351-356, 1992. R. Uebbing, “An Automated Digital Approach for the Generation of Digital Terrain Models Using HRSC and WAOSS Image Data of the Mars96 Mission“. International Archives of Photogrammetry and Remote Sensing, Vol. 31, Part B3, pp. 894-897, 1992. F. Wewel, “Determination of Conjugate Points of Stereoscopic Three Line Scanner Data of the Mars96 Mission“. International Archives of Photogrammetry and Remote Sensing, Vol. 31, Part B3, pp. 936-939, 1992. F. Wewel, F. Scholten, G. Neukum, J. Albertz, “Digitale Luftbildaufnahme mit der HRSC Ein Schritt in die Zukunft der Photogrammetrie“. Photogrammetrie-Fernerkundung-Geoinformation, 6, E. Schweizerbart’sche Verlagsbuchhandlung, Stuttgart, Germany, pp. 337-348, 1998.
