... Jr.z, Michael I. Millery,. Joseph A. O'Sullivan , Je rey H. Shapiro? , Donald L. Snyder ... The Johns Hopkins University,. Baltimore ... St. Louis, Missouri 63130 ?
Multispectral Sensor Fusion for Ground-Based Target Orientation Estimation: FLIR, LADAR, HRR Joseph Kostakisy , Matthew Cooper, Thomas J. Green, Jr.z, Michael I. Millery , Joseph A. O'Sullivan, Je rey H. Shapiro?, Donald L. Snyder Army Center For Imaging Science
y Department of Electrical and Computer Engineering
The Johns Hopkins University, Baltimore, Maryland 21218
Department of Electrical Engineering
Washington University, St. Louis, Missouri 63130
? Massachusetts Institute of Technology
Department of Electrical Engineering and Computer Science Cambridge, Massachusetts 02139-4307 z Massachusetts Institute of Technology
Lincoln Laboratory Lexington, Massachusetts 02420-9108
ABSTRACT
In our earlier work, we focused on pose estimation of ground-based targets as viewed via forward-looking passive infrared (FLIR)1 systems and laser radar (LADAR)2 imaging sensors.3 In this paper, we will study individual and joint sensor performance to provide a more complete understanding of our sensor suite. We will also study the addition of a high range-resolution radar (HRR).4 Data from these three sensors are simulated using CAD models for the targets of interest in conjunction with XPATCH range radar simulation software, Silicon Graphics workstations and the PRISM infrared simulation package. Using a Lie Group representation of the orientation space and a Bayesian estimation framework, we quantitatively examine both pose-dependent variations in performance, and the relative performance of the aforementioned sensors via mean squared error analysis. Using the Hilbert-Schmidt norm as an error metric,6 the minimum mean squared error (MMSE) estimator is reviewed and mean squared error (MSE) performance analysis is presented. Results of simulations are presented and discussed. In our simulations, FLIR and HRR sensitivities were characterized by their respective signal-to-noise ratios (SNRs) and the LADAR by its carrier-to-noise ratio (CNR). These gures-of-merit can, in turn, be related to the sensor, atmosphere, and target parameters for scenarios of interest. Keywords: Lie Groups, Automatic Target Recognition (ATR), Conditional Mean Estimation, Sensor Fusion, Laser Radar, Forward-looking Infrared (FLIR), High Resolution Radar (HRR). To
appear in the proceedings of the SPIE Conference on Automatic Target Recognition IX, Vol. 3717, Orlando, Florida, April 1999.
1. INTRODUCTION
Coherent laser radar (LADAR) systems can collect 2-D intensity, range, or Doppler images by raster scanning a eld of view. This imagery, however, is degraded by target speckle, atmospheric turbulence, and radar-beam jitter, as well as nite carrier-to-noise ratio (CNR). Of these factors, nite CNR and target speckle are the most signi cant for CO2 laser radars with modest-sized optics, i.e., 5{20 cm diameter optics. As a result, they have been the subject of careful modeling and analysis and are now reasonably well understood at the single pixel level.7 Many multifunction laser radar systems augment their active-sensor channel with a forward-looking infrared (FLIR) channel. We extended our sensor suite to include a high range-resolution radar (HRR), since modern signal processing techniques and the use of wideband radar technology have made it possible to resolve features of targets in their range pro les. The increased resolution capability has provided the means to extend the original concept of radio detection and ranging to include high resolution mapping and target pose and type identi cation. The problem we shall consider is estimation of the orientation of a target|when its class and position are known| using imagery and range pro les from such a sensor suite. Speci cally, we focus on the performance bene ts that accrue when multidimensional data are combined in a Bayesian estimation framework. The estimation algorithm uses three independent data sources: a LADAR range image, a FLIR intensity image and an HRR range pro le. The performance of each sensor is developed via simulation and compared to the performance obtained by optimally combining sensor measurements together. In all the imaging sensor cases, the images used were 64 64 pixels. In the case of the HRR, the range pro les had 151 range bins. The Bayesian estimation framework naturally accommodates the use of multiple sensors and allows us to quantify performance gains due to sensor fusion. Bayesian inference is based on the construction of the posterior density from the prior probability density of the underlying true scene parameter and the likelihood of obtaining the data. Speci cally, for a true scene parameter X 2 X , and data D1 ; D2 ; :::; Dn comprising n independent observations of the scene, the posterior density is the normalized product, :::; Dn j X ) ; for X 2 X , ( X j D1; D2 ; :::; Dn ) = (X )ZL((DD1;;DD2;;:::; (1) Dn ) 1 2 of the prior (X ), and the likelihood function L( D1 ; D2 ; :::; Dn j X ). The normalizing factor, Z (D1 ; D2 ; :::; Dn ), then equals the unconditional probability density for the data. In this paper, we use n = 3, for our LADAR, FLIR and HRR data.
2. PERFORMANCE ANALYSIS FOR POSE ESTIMATION
Our objective is to develop performance bounds|in the minimum mean-squared error (MMSE) sense|for pose estimation and to provide insight into the behavior of those bounds as di erent sensor suites are considered. Our approach is to derive the MMSE (the Bayes least-squares) estimator for each sensor suite under the assumption that all poses are equally likely. We then develop performance predictions for our estimators via Monte Carlo simulation. Our error metric, the mean-squared error (MSE), results as a special case of a more general : metric, the HilbertSchmidt norm.5,6,8 Namely, our parameter space is SO(2) and we denote the parameter as X = O 2 SO(2), where O is a 2 2 orthogonal rotation matrix with determinant 1. The MMSE estimator is de ned as follows: (D) = ArgMin E ( kO ? O^ k2 j D ) ; OHS (2) HS O2SO(2) Z O^ ( O^ j D ) (dO^ ) SO (2) (3) = v !: u Z u tdet O^ ( O^ j D ) (dO^ ) SO(2) To calculate the mean squared error (MSE), let OT describe the true underlying target orientation. Then, the unconditional MSE may be computed by evaluating: 2 MSE = E [E | ( kOT ? O{zHSkHS j OT })] : MSE(OT )
(4)
FLIR/LADAR Receiver Aperture Dimension 13 cm \ Receiver eld-of-view, R 2.4 mrad \ Detector Quantum Eciency 0.25 \ Atmospheric Extinction Coecient 0.5 dB/km FLIR Noise-Equivalent Temperature NET 0.1 K LADAR Average Transmitter Power 2W \ Pulse Repetition Frequency 20 kHz \ Pulse Duration 200 nsec \ Peak Transmitter Power 500 W \ Photon Energy 1.87 x 10?20 J \ IF Filter Bandwidth 20 MHz \ Range Resolution 30 m \ Number of Range Bins 256 HRR Center Frequency 1.5 GHz \ Bandwidth 739.5 MHz \ Frequency Sample Spacing 4.921 MHz \ Number of Frequency Samples 151 \ Azimuth Angle Spacing 0.15 \ Number of Azimuth Samples 2401
Table 1. System parameters for the FLIR, LADAR and HRR sensors. Using the discrete set of orientations = O1 ; :::; OM 2 SO(2) and assuming a uniform prior on the underlying target orientation Om , the MSE is found by rst calculating the conditional mean square error, MSE(Om ), and then averaging this conditional over O: M X MSE = 1 MSE(O ): (5)
M m=1
m
The conditional MSE(OT ) is evaluated using numerical approximations to the integral over SO(2) and Monte Carlo random sampling to compute the expectation over D, the data measurement space. The noise environment for each simulation run is chosen to produce the FLIR and HRR signal-to-noise ratio (SNRF ; SNRH ) and LADAR carrierto-noise ratio (CNR) values that are desired for that run. Thus, we generate N data sets, D1 ; :::; DN , at noise environment (SNRF ; SNRH ; CNR) for each orientation of the true target OT , and then compute an estimate of MSE(OT ) according to N X 1 d kO (D ()) ? O k2 : (6) MSE (O ) = SNRF ;SNRH ;CNR
d OT ) ! MSE(OT ) as N ! 1. Notice that MSE(
T
N n=1
HS
n
T HS
3. STATISTICAL SENSOR MODELS
Our ground-imaging sensor suite consists of a long-wave infrared laser radar capable of collecting range information, a FLIR system and an HRR sensor. The LADAR and FLIR imaging subsystems share optics and are thus pixelregistered; that is, for each pixel of active data there is a corresponding pixel of passive data simultaneously recorded. The sensor suite is airborne, ying at an altitude of 932 meters and is looking forward at an angle of 25 degrees and a ground distance of 2.0 kilometers. The distance between the sensor and the target area is approximately 2.21 kilometers. The sensor characteristics we shall employ, summarized in Table 1, are based on the parameters for the sensor suite of MIT Lincoln Laboratory's Infrared Airborne Radar (IRAR) Program.9 For the FLIR sensor, we assume that target facets and the background radiate known intensities; for the LADAR sensor, we assume that the range to target is known; for the HRR sensor, we assume that the range pro le of the target is known. In the simulations that follow, the range, the intensities and the range pro les are predicted by
a ranging process, a rendering process and a pro ling process implemented on a Silicon Graphics Onyx2/In nite Reality Engine. Samples of these processes can be seen in Figure 1 and 2.
Figure 1. Left: The CAD model for the T62 Russian tank. Top: Left panel shows the noise-free LADAR range image with optical projection, and right panel shows the image corrupted by noise. Bottom: Left panel shows the noise-free FLIR intensity image, and right panel shows the image corrupted by noise. Range profile for a t1 tank at 60° azimuth
Range profile for a t1 tank at 60° azimuth
0.14
0.4 10 dB 0.35
0.12
0.3 Range Profile Magnitude (m2)
Range Profile Magnitude (m2)
0.1
0.08
0.06
0.25
0.2
0.15
0.04 0.1
0.02
0
0.05
0
20
40
60
80 Range Bins
100
120
140
160
0
0
20
40
60
80 Range Bins
100
120
140
160
Figure 2. Panel 1 (left) shows the T1 tank used in the URISD data; panel 2 (middle) depicts the ideal range pro le for the target; panel 3 (right) depicts the range pro le corrupted by noise. The eld of view of the optical telescope of the imaging sensors employed in our scenario requires the use of perspective projection, in which a point (x; y; z ) in 3-D space is projected onto the 2-D detector by the simple mapping (x; y; z ) 7! (x=z; y=z ). This creates the vanishing point e ect in which objects that are farther away appear closer to the center of the detector. Objects appear skewed in di erent ways depending on where they appear on the image plane. We assume that the image lattice L for the scene is a 64 64 array of distinct elements ` 2 L and the range pro le of the target scene is a one-dimensional array with 151 distinct elements } 2 P . Although a number of points in 3-D space may map to the same place on the detector under projection, only the point closest to the detector contributes to the detector output, since targets (and background) are opaque at the optical wavelength under consideration.
Thus, objects in the scene may partially or totally obscure other objects. For this reason, we use functions of target orientation render(O), range(O) and profile(O) to represent the operations of \rendering" the radiant intensities, \ranging" the range information and \pro ling" the range pro le, respectively, of the objects and background in the scene|via perspective projection and obscuration|onto the image plane. Implicitly, render, range and profile are functions of the airborne platform's ight path. The rendering, ranging and pro ling processes can be quite intricate; thus the log-likelihood is a highly non-linear function of the target, orientation, and channel parameters. The FLIR, the LADAR and the HRR are assumed to be statistically independent of each other, and the ideal images and range pro les corresponding to each of these sensors are corrupted according to the di erent noise models discussed below.
3.1. Active-Imager Statistics
Let dR =: f dR (`) : ` 2 L g be the random range image produced by the LADAR, and let dR =: f dR (`) : ` 2 L g be the associated ideal range image for the scene. Neglecting the digitization found in typical laser radars, dR is modeled as a continuous-parameter random process with conditional probability density (PDF), given the ideal range image dR , given by,2 pdRjdR ( DR j DR ) =
(D (`) ? D (`))2 0 1 R R exp ? 2 Y BB C; p 2D2 + Pr(A) C @[1 ? Pr(A)] A
`2L
2D
D
for DRmin DR ; DR DRmax :
(7)
Here, Pr(A) is the single-pixel anomaly probability|the probability that speckle and shot-noise e ects combine to yield a range measurement that is more than one range resolution cell from the true range|which we assume to be the same for all pixels. The rst term in the product on the right in Eq. 7 represents the local range behavior for pixel `. It equals the probability that pixel ` is not anomalous times a Gaussian probability density with mean equal to the true range DR (`) and standard deviation equal to the local range accuracy D. The second term in the procuct on the right in Eq. 7 represents the global range behavior of pixel `. It equals the anomaly probability times a uniform probability over the entire range-uncertainty interval, DRmin DR ; DR DRmax , which is assumed to include the true range DR and to have an extent, D DRmax ? DRmin, that is much larger than the local accuracy D.
3.2. Passive-Imager Statistics
For the FLIR sensor, we assume high-count-limit direct detection and adopt a white-Gaussian random process model for the intensity image dF =: fdF (`); ` 2 Lg, conditioned on the ideal intensity image dF =: fdF (`); ` 2 Lg. Thus, the likelihood function for the FLIR takes the following form, 0 (DF (`) ? DF (`))2 1 2 Y B exp ? CC : q 22F (8) pdF jdF ( DF j DF ) = B A @ 2F `2L
3.3. High Range-Resolution Radar Statistics
For the HRR sensor, we adopt the additive complex white Gaussian noise model for the complex for the : ffdHenvelope = ( } ) ; } 2 Pg be observed signal and hence the magnitude of the observed signal is Rice distributed. Let d H the random range pro le produced by the HRR and let dH =: fdH (}); } 2 Pg. Then, the likelihood function has the following form, DH (})2 + jD (})j2 Y 2 DH (})jDH (})j DH (}) H pdH jdH ( DH j DH ) = 2 I0 (9) 2 +1 2 + 1) exp ? 2 + 1) ( } ) ( ( } ) ( ( } ) }2P where I0 is the zeroth-order modi ed Bessel function of the rst kind.
3.4. Fusion
The ideal scenes of Figure 1 and 2 are corrupted by various sources of error as discussed in the sections above. For the imaging sensors, the data space L is a rectangular lattice and for the third sensor, the data space P is a one dimensional array corresponding to the sizes and shapes of the detector scan. The sensors are assumed independent, so that the joint log-likelihood is the sum of the log-likelihoods of each sensor. The Gaussian log-likelihood for the FLIR sensor at each target orientation O is used with the mixed Gaussian and uniform log-likelihood for the LADAR sensor and the Rician log-likelihood for the HRR sensor given the ideal observed intensity DF , range DR and range pro le DH :
L( DF ; DR ; DH j O ) = L( DF j O ) + L( DR j O ) + L( DH j O )
q = ?(64)2 ln( 2F2 ) ? 12 F
+ +
X `2L
[DF (`) ? DF (`)]2
(D (`) ? D (`))2 8 2 39 R R > > exp ? 64[1 ? Pr(A)] D 5> 2D2 :`2L ; X ( 2 D (})jD (})j D D (})0 2 + jD (})j2 ) }2P
ln 2 I0
H
(10)
H
(})2 + 1
H H H ((})2 + 1) ? ((})2 + 1)
(11)
:
(12)
4. RESULTS ON POSE ESTIMATION
The target considered herein for pose estimation is a CAD model for a T62 Soviet tank for the imaging sensors, with the T1 tank used in the URISD dataset. The simulation program evaluates the posterior probability for the target pose conditioned on the observation. Then, it computes the Hilbert Schmidt estimate for the pose. The computation is repeated evaluating the MSE according to Eq. 5. The process is repeated for each integer orientation in the quarter circle, = fO0 ; :::; O89 g, where Oi is the rotation matrix corresponding to a rotation of i = i degrees, and for each noise level considered. For the active sensors, LADAR and HRR, the noise environment is controlled by varying the respective transmitter powers. For the passive sensor, the noise level is controlled by varying the noise-equivalent di erential temperature of the detector. The pixels on target for the active and passive imagers are the same for each orientation and vary with the target orientation as can be seen in the left of Figure 3. The simulations were performed by xing CNR and SNR's as described below. The top and bottom row of Figure 3 depict simulated imagery for the LADAR and FLIR sensors at di erent noise levels.
4.1. Active-Imager Statistics
The statistical model for the LADAR data preserves the essential features of a ne-range pulsed imager, a system capable of 3-D imaging. The local accuracy and range anomaly behavior incorporated in Eq. 7 have been established through theory, simulation and experiment in Shapiro et al.10 In terms of the carrier-to-noise ratio, radar-return power (13) CNR = averageaverage local-oscillator shot noise power ; the range resolution Rres |roughly :cT=2 for a pulse duration of T seconds, where c is the speed of light| and the number of range p resolution bins N = D=Rres , we have that the local range accuracy and anomaly probability obey D Rres = CNR and 1 1 Pr(A) CNR ln(N ) ? N + 0:577 ; (14)
respectively. These results suce for the interesting regime of N 1 and CNR 1; more exact results are available if other regimes need to be considered.10
T62 Analysis of Orientations 360 Flir−Ladar 340
Pixels on target
320
300
280
260
240
220
0
10
20
30
40 50 Orientation
60
70
80
90
Figure 3. Left: Pixels on target for each orientation in the quarter circle. Top: Simulated LADAR imagery for 25 dB CNR (left panel) and 30 dB CNR (right panel) at 60 degree orientation. Bottom: Simulated FLIR imagery for 5 dB SNR (left panel) and 10 dB SNR (right panel) at the same orientation. The CNR, for the typical case of monostatic operation, is given by the resolved speckle-re ector radar equation:11
PT AR " exp(?2 D ): CNR = hB R D2 R
(15)
Here: is the receiver's photodetector quantum eciency; PT is the radar transmitter's peak power; h is the photon energy at the radar's laser wavelength, = c= ; B is the radar receiver's IF bandwidth; is the re ectivity for the pixel under consideration; AR is the radar receiver's entrance-pupil area; " is the product of the radar's optical and heterodyne eciencies; and is the atmospheric extinction coecient, which is assumed to be constant along the propagation path.
4.2. Passive-Imager Statistics
For the passive channel, the integrated intensity measured in each pixel, normalized to units of power, can be written as:12 p^ = Ps + ns ; (16) where s = t or b for target or background, respectively, Ps is the radiation power (target or background) incident on the photodetector and ns is a hypothesis-dependent, zero-mean, Gaussian-distributed, shot-noise-plus-thermal-noise random variable. Equation 16 assumes that the constant o set due to the dark current and excess background radiation has been subtracted from the photocurrent. The most common gure of merit used to describe thermal sensing systems is the noise-equivalent di erential temperature, NET , given approximately, for h kTS , by12
s
2 s 2Bp (Ps + Pd + Ptherm ); NET kT Ps h
(17)
where k is Boltzmann's constant, Ts is the absolute temperature of the source (target or background), is the passive channel's center frequency, Pd Id h=q is the dark current equivalent power for mean dark current Id , q is
the electron charge, and Ptherm is the thermal-noise equivalent power, Ptherm 2kTLh=q2 RL , with TL being the absolute temperature and RL the resistance of the photodetector's load. We have ignored any excess background radiation emanating from the passive-channel telescope. If we consider Pb to be the nominal radiation power in the eld of view, then we can think of P Pt ? Pb as the signal power present in a target pixel with di erential temperature T Tt ? Tb . The image signal-to-noise ratio is then written SNR (T=NET )2 . In other words, as the radar scans a scene, we are interested in sensing the weak temperature variations caused by the presence of a target masking a portion of the background. Physically, NET is the temperature di erence which produces unity SNR.
4.3. High Range-Resolution Radar Statistics The general model for the observation is
r(t) = s(t; O; a) + w(t) (18) where r(t) is the complex envelope of the observed range pro le, s(t; O; a) is the signal part of the waveform, O is the orientation, a is the target class, which is assumed to be known, and w(t) is additive noise. We consider a stochastic radar model under which, given the target orientation, the signal portion is s(t; O)
of the received range pro le forms a complex Gaussian random process. The conditional mean and covariance are denoted as m( t j O ) and K ( t1; t2 j O ), respectively. The wide-sense-stationary uncorrelated-scatter (WSSUS) model for di use radar targets described by Van Trees13 assumes that returns from di erent delays are statistically uncorrelated. For the conditionally Gaussian model, we extend this assumption to samples of the range pro les. Let the model be discretized such that samples of the received range-pro le r(t) are collected in the column vector r. Then the model is r = sO + w ; (19) where sO is a Gaussian random vector that has mean mO and covariance matrix KO , and w is another Gaussian random vector that has mean 0 and covariance N0I, where I is the identity matrix. The signal sO and the noise w are assumed statistically independent. Thus,
E f r j O g = mO E f [r ? mO ]y [r ? mO ] j O g = KO + N0 I;
(20) (21)
where y denotes the complex conjugate transpose. Under this model, the likelihood of observing jrj, given O, is Rice distributed. The key assumption of the conditionally Gaussian model is that the returns from di erent range bins are statistically uncorrelated, so that KO is a diagonal matrix. For K bins for each range pro le, sO = fsO (h) : 1 h K g, where sO (h) is a Gaussian random variable with mean mO (h) and variance O2 (h) for 1 h K. Let us then de ne: 2 (22) 0 2 (h) = O (h) O
2 m0O (h) = mO(h)
(23)
average target radar cross-section SNR = noise-equivalent radar cross-section
(24)
K K (jmO (h)j2 + 2 (h)2 ) X X 0 0 Oref ref SNR = K1 (jmOref (h)j2 + O2ref (h)) = K1 2
(25)
Let us de ne the signal-to-noise-ratio as
Then, the SNR is given by h=1
h=1
where Oref is a reference angle,chosen such that the average target cross-sectional area for this angle is maximum.
4.4. Simulation Results
The simulation program produced performance curves for the individual sensors and for multiple sensors optimally combined in the Bayesian sense. Panel 1 of Figure 4 depicts the Hilbert-Schmidt bound performance curve for the LADAR sensor. Panel 2 depicts the performance curve for the FLIR sensor and panel 3 the performance curve for the HRR sensor. For the individual sensors, the MSE increases as the noise increases. Panel 4, depicts the HSB performance curves for the LADAR and HRR under a joint noise level axis. Assuming that we could control the noise level of the two sensors, LADAR and HRR, in a xed ratio by controlling the power of both sensors in a xed ratio, then the dash-dotted line represents the performance bound for the two sensors optimally fused together under such a combination of their noise levels. Unconditional HSB Performance Curve for FLIR 25
0.4
Unconditional HSB Performance Curve for Long Pulse LADAR
FLIR
25
0.4 LADAR
0.35
21 0.25
MSE
19 0.2 16 0.15 13 0.1
9.
0.05
0
8
10
12
14
16
18
20
22
24
23
0.3 21 0.25 19
MSE
0.3
Equivalent Estimation Error In Degrees
23
0.2 16 0.15 13 0.1
9.
0.05
0 −40
0
Equivalent Estimation Error In Degrees
0.35
−35
−30
−25
−20
−15
−10
−5
0
5
10
0
SNR (dB)
CNR (dB)
Unconditional HSB Performance Curve for HRR 25
0.4
Unconditional HSB LADAR, HRR and Joint Performance Curve
HRR
21 0.25
MSE
19 0.2 16 0.15 13 0.1
LADAR HRR JOINT
0.35
23
0.3 21 0.25 19 0.2 16 0.15 13
Equivalent Estimation Error In Degrees
0.3
Equivalent Estimation Error In Degrees
23
MSE
0.35
0.1
9.
0.05
0 −30
25
0.4
9.
0.05
−20
−10
0
10
SNR (dB)
20
30
40
50
0
0
5
10
15
CNR (dB), 0.3462 × SNR + 13.23 (dB)
20
25
0
Figure 4. Panel 1 (top left) shows the Hilbert Schmidt bound (HSB) curve for the LADAR; panel 2 (top right) shows the HSB curve for the FLIR; panel 3 shows the HSB curve for HRR; panel 4 shows a joint LADAR and HRR 2-D HSB performance curve.
Using information from two sensors the performance gain that accrues from sensor fusion can be visualized as the decrease of SNR's or CNR required to achieve the required performance level. Panels 1, 2 and 3 of Figure 5 show (CNR; SNR) requirements needed to realize a xed MSE value of 0.05, viz. a root-mean-square (rms) pose estimation error of 9 . For panel 1, the horizontal straight line is the HRR SNR required to achieve the desired pose-estimation performance from the HRR sensor alone, and the vertical straight line in this panel is the LADAR CNR required for the active LADAR channel alone to achieve the desired pose-estimation performance. The solid fusion curve in panel 1 shows the (CNR; SNRH ) values needed to realize the desired pose-estimation performance when the HRR and LADAR outputs are optimally combined in our Bayesian framework. The dashed fusion curve shows (CNR; SNRH ) values needed to realize the same pose-estimation performance with the FLIR sensor operating
at SNRF = ?17 dB. Panels 2 and 3 in Figure 5 have interpretations similar to that just described for panel 1. In particular, the horizontal and vertical lines in these panels are single-sensor requirments for achieving 9 rms pose estimation error, the solid curves are the two-sensor-fusion requirements for achieving this pose estimation performance, and the dashed fusion curves show the additional performance bene t|in achieving 9 root-mean-square error|when the third sensor is added at a xed performance level. Thus the solid curve in panel 2 is the (CNR; SNRF ) contour for 9 rms error and the solid curve in panel 3 is the (SNRH ; SNRF ) contour for 9 rms error under our optimal Bayesian fusion framework. Likewise the dashed curves in panels 2 and 3 are the changes in these contours resulting from the respective addition of an HRR sensor operating at SNRH = ?2 dB and a LADAR sensor operating at CNR = 12 dB to our optimal Bayesian fusion. These results quantify the parameter-regime extensions that accrue as additional sensors are added into an optimal fusion framework. HSB Performance Curve for 9 degrees of error
HSB Performance Curve for 9 degrees of error
HSB Performance Curve for 9 degrees of error
12
−10
10
−11
8
−12
6
−13
−10
2
0
−14
FLIR SNR (db)
4
FLIR SNR (db)
HRR SNR (db)
−12
−14
−15
−16
−2
−17
−4
−18
−16
−18
−20
−22
−6
−8
−19
FLIR=−17 dB FLIR= −∞ 9
10
11
12
13
LADAR CNR (dB)
14
15
16
17
−20
LADAR=12 dB LADAR=−∞
HRR=2 dB HRR= −∞ 8
9
10
11
12
13
14
15
16
17
−24 −2
LADAR CNR (dB)
0
2
4
6
8
10
12
HRR SNR (dB)
Figure 5. Panel 1 (left) shows joint LADAR and HRR performance-bound curves for 9 root-mean-square pose estimation error with and without the contribution of the third sensor (FLIR); panel 2 (middle) shows the joint FLIR and LADAR performance curve with and without any contribution from HRR; panel 3 (right) shows the joint HRR and FLIR performance curve with and without any contribution from the LADAR sensor.
5. CONCLUSION AND FUTURE WORK
The simulation results show the pose-estimation performance bound for the LADAR, FLIR and HRR systems. Two sensors optimally fused perform better than the individual sensors at the same noise levels and equally well in lower noise levels as depicted in the panels of Figure 5. The addition of a third sensor at some signi cant CNR or SNR improves the performance of that of a two-sensor suite. This progressive performance improvement|with each additional sensor|is to be expected, because we are doing optimal Bayesian fusion and there is more information available in the joint posterior distribution of two sensors than in the posterior distribution of either single sensor, and more information available in the posterior distribution of three sensors than in the posterior distribution of any two of these sensors. In future work, we will pursue the addition of a video imaging sensor in our sensor suite to perform sensor fusion in the Bayesian estimation framework for pose estimation on ground-based targets.
6. REFERENCES
[1] M.L. Cooper, U. Grenander, M.I. Miller and A. Srivastava, \Accommodating Geometric and Thermodynamic Variability For Forward-Looking Infrared Sensors," Proc. SPIE 3070, 162{172 (1997). [2] T.J. Green, Jr., and J.H. Shapiro, \Detecting Objects in Three-Dimensional Laser Radar Range Images," Opt. Eng. 33, 865{874 (1994).
[3] J. Kostakis, M. Cooper, T.J. Green, Jr., M.I. Miller, J.A. O'Sullivan, J.H. Shapiro and D.L. Snyder, \Multispectral Active-Passive Sensor Fusion for Ground-Based Target Orientation Estimation," Proc. SPIE 3371, 500{507 (1998). [4] S.P. Jacobs and J.A. O'Sullivan, \High Resolution Radar Models for Joint Tracking and Recognition," IEEE Proc. National Radar Conf., pp. 99{104 (May, 1997). [5] U. Grenander, M.I. Miller, and A. Srivastava, \Hilbert-Schmidt Lower Bounds for Estimators on Matrix Lie Groups," IEEE Trans. on Pattern Analysis and Machine Intell., 20(8), 1{13, (August, 1998). [6] A. Srivastava, U. Grenander, and M.I. Miller, \Ergodic Algorithms on Special Euclidean Groups for ATR Systems and Control in the Twenty-First Century," in Progress in Systems and Control, Vol. 22 (Birkhauser, 1997). [7] S.M. Hannon and J.H. Shapiro, \Active-Passive Detection of Multipixel Targets," Proc. SPIE 1222, 2{23 (1990). [8] A. Srivastava, \Inferences on Transformation Groups Generating Patterns on Rigid Motions," Ph.D. thesis, Washington University, August 1996. [9] J.K. Bounds, \The Infrared Airborne Radar Senor Suite," Research Laboratory of Electronics Tech. Report 610, MIT, December 1996, Table 1. [10] J.H. Shapiro, R.W. Reinhold, and D. Park, \Performance Analyses for Peak-Detecting Laser Radars," Proc. SPIE 663, 38{56 (1986). [11] J.H. Shapiro, \Target Re ectivity Theory for Coherent Laser Radars," Appl. Opt. 21, 3398{3407 (1982). [12] S.M. Hannon, \Detection Processing for Multidimensional Laser Radars," Ph.D. thesis, MIT, December 1989. [13] H.L. Van Trees, Detection, Estimation and Modulation Theory, Part III , John Wiley and Sons, New York, 1971.
