Joint kinematic and feature tracking using probabilistic argumentation Deepak Khosla HRL laboratories, LLC Malibu, CA
Yang Chen HRL laboratories, LLC Malibu, CA
[email protected]
[email protected] associate sensor measurements with existing or new tracks. This data association process is uncertain due to inaccurate measurements, partially obscured targets, closely-spaced target configurations, poor signal-to-noise ratio, random false alarms in the detection process, clutter near the target of interest, interfering targets, decoys or other counter measures, etc.
Abstract – This paper describes a multi-target tracking approach based on merging traditional Kalman recursive filtering with evidential reasoning methodology. The approach fuses all the available information about targets to be tracked – both kinematic (position, velocity, etc.) and identity (attributes, features, type, class, etc.) information - and uses the fused measure for data association and tracking. The proposed tracking system departs from other feature-aided tracking systems in that (1) it allows more flexible domain knowledge representation such as in the form of uncertain rules, (2) it is able to accommodate imprecise as well as partial knowledge (e.g., partial probability distribution), and (3) it uses a probabilistic argumentation system as the evidential reasoning methodology for attribute/feature fusion. The method described here is general purpose, will work on multiple-target tracking problems with both single and multi-sensor systems and is not sensor type dependent. We demonstrate the feasibility and utility of the proposed method through a multi-target tracking simulation example.
The state-of-the-art in MTT is a Kalman filter based multiple hypotheses tracker (MHT) that uses kinematic (position, velocity, etc.) information to carry out the association process. A multiple hypothesis tracking (MHT) data association method with interacting multiple model (IMM) filtering is described in [1] and has been applied to several real-world applications. This tracker uses kinematic (or metric) quantities and certain signalrelated quantities for data association. No use is made of features or attributes in data association. Both the memory and computation requirements of MHT increase exponentially with problem size. In most applications, the target identification or class information is obtained by using attribute or feature measurements outside of the kinematic data association module. Thus these are completely uncoupled and independent processing. While this approach works well for relatively benign environments, it performs poorly in difficult environments. For example, it defaults to group tracking in dense track environments, or loses track during complex movements or over long periods of time. This could result in poor track and attribute estimation, lost tracks, misidentified objects, slow system response, etc.
Keywords: Tracking, Feature-aided tracking, Kalman, filter, fusion, probabilistic argumentation, evidential reasoning, rule-based system
1
Introduction
Multiple Target Tracking (MTT) is an essential requirement for surveillance, tracking and control systems employing one or more sensors in order to interpret an environment that includes both true objects and false alarms. The goal of tracking is to robustly provide accurate and timely information on the number of objects and their state and type. This is critical in both military and commercial applications. A typical military application is automated target tracking and identification of ground and aerial targets. An example in commercial application is in automotive forward collision warning, which requires both kinematic and type (roadside/vehicle/pedestrian) information on in-path obstacles for effective system performance. The central problem of multiple target tracking is the problem to
Major improvements to the state-of-the-art are needed, but can only be achieved by a paradigm shift in approaches to the MTT problem. In this paper, we describe a feature-aided tracking approach that incorporates a traditional Kalman recursive filter within an evidential reasoning framework. In deciding whether to associate measurement data with new or existing tracks, the method uses all the data supporting them during their common history. The goal is to take as input raw measurements or tracks of both positional and attribute information and produce as output all-source tracks by associating the measurements/input tracks, and
11
PAS theory can be found in [6,7]. The idea behind PAS is that the knowledge about a process or system may be uncertain, incomplete, or even inconsistent. The uncertainty of the knowledge is expressed by assumptions attached to the rules that model this system. Given such a knowledge base, the problem is then to find arguments for hypotheses (queries). Arguments are built from the assumptions appearing in the model and can be calculated through a resolution process similar to those used in predicate logic. Arguments for which the hypothesis is certainly true are called supports (or quasisupports). Arguments for which the hypothesis is certainly false are called doubts, and arguments for which the hypothesis is possible (or plausible) are called plausibilities. One can then compute sets of supports, doubts, etc. Its applicability to identity estimation or fusion in the proposed method is to obtain sets of support for certain target type or disjunction of target types based on the knowledge and new evidence (measurements).
fusing the corresponding kinematic and attribute state information in a single step. The advantage of unified estimation and decision is that it can handle both continuous and discrete uncertainties and use information across modalities for improved accuracy. The method uses a joint state that comprises the Gaussian kinematic state and the discrete states, which are attributes or features extracted from the measurements. The use of features/attributes in the data association will increase the robustness of the tracker to missing/incomplete/uncertain measurements. Several studies in the literature [4,5] show track quality improvement through the use of features in the data association process. In this work, we present a feature-aided tracking approach that utilizes feature/attribute metrics to increase the accuracy and robustness of data association. In order to determine the feature metrics, we first need to model the system and encode information about the target type/class, feature/attribute types, relationship between target type/class and features/attributes, etc. The modeling language used in this work is based on “uncertain rules” [6,7]. Using this domain knowledge and incoming feature/attribute measurements, evidential reasoning methodology can be used to determine target type/class information. This information is used in the data association step. This part of the problem is similar to a target classification problem and is treated as “reasoning under uncertainty” because the knowledge and measurements can be both uncertain and incomplete. In this work, we use a probabilistic argumentation system (PAS) as the reasoning engine.
The linkage between PAS and D-S is that the quantities such as support and plausibility of hypotheses obtained from the PAS are the same as the corresponding quantities in a D-S system if the rules in the knowledge base are represented in the D-S system appropriately. However, the advantages of using PAS rather than D-S are that the system modeling process is more flexible and incremental. There is also a potential saving in computation resource without using a full D-S model for a specific application.
Probabilistic argumentation system (PAS)
2
Probabilistic argumentation system (PAS) [6,7] is a powerful technique for solving problems consisting of uncertain, incomplete or inconsistent information and is based on a novel combination of classical logic (for deduction) and probability theory (for measuring the reliabilities of deductions). The PAS approach of dealing with partial information is closely connected to the Dempster-Shafer (D-S) Theory of Evidence [3]. PAS and DS are both able to accommodate partial knowledge (e.g., partial probability distribution) and imprecise knowledge. They address the real-world limitations of knowledge by generalizing Bayesian probability calculus. If knowledge can be provided at the level of specificity of a probability distribution for each feature for each class, as is necessary for Bayesian probability fusion [8], then DS, PAS and Bayesian results are the same.
2.1
Method Fusion functional block diagram
The proposed approach is a tracking system that combines a Kalman filter (Interacting Multiple Model (IMM)) and a Probabilistic Argumentation System (PAS). The Interacting Multiple Model (IMM) estimator is a hybrid filter that is able to estimate the state of a dynamic system with several behavior modes - which makes it natural for tracking maneuvering targets. Probabilistic argumentation system (PAS) is a powerful technique for solving problems consisting of uncertain, incomplete or inconsistent information and is based on a novel combination of classical logic (for deduction) and probability theory (for measuring the reliabilities of deductions). The inputs to the PAS are attributes or features or target types and their uncertainties. The PAS outputs are feature-track likelihood scores that are combined with kinematic-track likelihood scores for improved tracking by the Kalman filter. This association step is the key advantage of the proposed method, where the likelihood is a joint measure over the continuous and
However, PAS has the added advantage that the default knowledge base and measurements are expressed in the form of logical rules and clauses making it very amenable to most rule based forms of knowledge. Details about the
12
methods, the basic premise is to use the kinematic measurements in a score such as (1) and use it for data association (i.e., which measurement belongs to which existing track or a new track).
discrete space, and is expected to dramatically increase the accuracy of the association step and the robustness of the tracker to missing/incomplete/uncertain measurements. Knowledge Base (Rules, probabilities, ..) Track file (Track number, position, velocity, ID)
Kinematic measurements
Kinematic likelihood scores
Probabilistic argumentation system
2.2.2 Extension to feature data Feature/class measurements
Feature likelihood scores
Joint track likelihood scores
with this track Zˆ , the probability that this track produced measurement Z is given by
Trackmeasurement association Track update
J
P( Z / Zˆ ) = ∑ P( Z / T j ) P(T j / Zˆ )
(2)
j =1
Figure 1 Block diagram
2.2
In order to make use of available attribute/feature/type information about the targets in addition to kinematic information, we need to create a measure that incorporates both of these information types. We therefore need to compute the probability term associated with the potential assignment of an attribute or features to a given track. Assume that an identity or type estimate has been formed from previous measurements assigned to this track. Then given the present type measurement Z and the set of previous type measurements associated
assuming the measurements are independent given the target type. Here Tj are the target ID or types that the measurement Z can take. Often instead of the track type information being available directly, the feature information (F) about the target type is available. In such a case one needs to know the relation between the feature and the target type as well. The above equation would then be modified to:
Fusion details
2.2.1 Kinematic measurement based association In this work, we use a global nearest neighbor (GNN) approach for data association. This is often referred to as single hypothesis tracking or sequential most probable hypothesis tracking. It handles the data in a purely sequential manner. For each new measurement, the goal is to identify the most likely assignment of observations to existing tracks and to the new source hypothesis. Observation-to-track assignment scores are computed by using the generalized statistical distance [1] as:
assuming the feature measurement is independent of the target type given the true feature. Incorporating this type or feature probability measure into the normalized distance in (1), we get:
d G2ij = d ij2 + ln S ij
d G2ij = d ij2 + ln Sij − 2 ln P( Z / Zˆ )
J
P ( Z / Zˆ ) = ∑ P( Z / F ) P( F / T j ) P(T j / Zˆ )
(3)
j =1
[
(1)
where Sij is the residual covariance matrix and d2ij is the normalized statistical distance (defined in terms of residual between prediction for track i and observation j) are standard Kalman filtering terms [1].
]
(4)
2.2.3 PAS for feature likelihood term The above approach shows one method to include feature/attribute data in the data association step using a Bayesian formulation. One limitation of Bayesian methods is that the conditional probabilities need to be known for feature/attribute measurements. Often such probability descriptions are not available, or are incomplete. Probabilistic argumentation systems and evidential reasoning methods are both ideal in this
The above is the cost for those observation-to-track assignments that satisfy the gate. An arbitrary large cost can be assigned to pairings that do not satisfy the gate. Other methods use a multiple hypothesis tracking loglikelihood or score function similar to [2]. In all of these
13
actual targets. In addition, the measurement errors are sometimes larger then the separation between the targets (clearly seen in the zoomed view).
situation since they do not require complete probablistic information. Furthermore, they have a distinct advantage here in that a quantity called the degree of contradiction (k) can be used in lieu of the probabilistic measure [9]. The degree of contradiction k in a PAS system is a measure of conflict or contradiction between the current knowledge state and the measured attributes/features. The
( ) for the Bayesian
quantity 1−k is similar to P Z Zˆ
approach and is a measure of the correctness of a given association [9]. Substituting this in (4) gives Figure 2 Simulated target track trajectories
d
2 Gij
= d + ln Sij − 2 ln[1 − k ] 2 ij
(5)
Let us assume that each aircraft is of a different type (ID) and domain of target types or classes is {T1, T2, T3}. Assume that we have a discrete feature F of domain {F1, F2, F3}. Let us assume that the following domain knowledge (incomplete probability specification) about the target-feature relationship is available to us in the form of rules relating target type with declared feature class.
2.2.4 Track pairing The above normalized distance is a score based on both kinematic and attribute/feature measurements. Note that this score will be computed for all candidate trackmeasurement pairings. In order to reduce feature processing, standard gating is used. Only observation-totrack associations that satisfy this gate are sent to the PAS for computing the degree of contradiction (k). The combined score forms the elements of a 2-D assignment matrix. We solve the assignment problem using the Munkres algorithm, though several other approaches could be used. The output from the assignment step is a track-measurement pairing. This information is then used to update the track file. Track kinematic parameters are updated using the standard IMM update equations. Track ID estimates are updated using PAS.
3
RULE
R1
T1 => F1;
80%
RULE
R2
T1 => F2 or F3;
10%
RULE
R3
T2 => F1;
5%
RULE
R4
T2 => F2;
75%
RULE
R5
T2 => F3;
25%
RULE
R6
T3 => F3;
90%
For example, Rule R1 states that there is an 80% probability of declaring feature F1 if the target is of type T1. Similarly Rule R2 states that there is a 10% probability of declaring features F2 or F3 from target type T1. This is a typical way to represent ignorance or uncertainty in a probabilistic argumentation system. Note that in a Bayesian framework, probabilities would have to be assigned only to singletons and any leftover probabilities would have to be assigned to leftover singletons and sum to unity. In the above knowledge base, we do not have such restrictions. In PAS, these rules would be appended with assumptions with the given probabilities.
Simulations
The current approach has been evaluated with case of tracking 3 maneuvering aircrafts in aerial combat. Figure 2 shows a 2D view of the actual target trajectories (solid colored lines). These targets are in aerial combat, fly in close proximity at times, and as seen maneuver towards or away from each other. In the zoomed inset in the lower left hand corner (at beginning of simulation), the targets are flying very close to each other. We assume that we have an adaptive radar sensor that provides position measurement every 3 sec during nonmaneuvering mode and every 0.3 sec during maneuvering mode. In actual implementation, a situation awareness module that uses the IMM mode probabilities as its inputs to determine the target mode controls the radar sampling rate. The colored squares are the position measurements made by the radar sensor. The radar only makes the measurements – it does not know which measurement is from which track or target. Because of the proximity of the targets, they are sometimes not resolvable by the radar resulting in number of measurements that are fewer than number of
Let us assume that we start with complete ignorance, i.e., no knowledge about the target class for each of the tracks. The standard state vector (position, velocity) representation has been augmented to include target ID estimates. For each simulation step, we use the kinematic measurements to compute the kinematic association score. The features measurements are added as new rules and fused using the PAS to obtain the updated target type probabilities and the feature likelihood scores. The data
14
association is carried out as described in sections 3.1 and 3.2. Based on the data association, the track state vectors are updated.
3.1
Table 1 (column labeled without features) shows the results based on use of only kinematic measurements from the radar. Using (1) for data association gives frequent mis-associations (23% incorrect trackmeasurement pairing). This results in poor track quality in terms of kinematic accuracy as indicated by the large RMS position error. If we use the data association results to update the track ID in the PAS module, we also get very poor ID estimation accuracy with 14% non-declared ID and 4% false track ID.
We use the following performance metrics to compare the tracker performance with and without feature data. These metrics are based on the entire time history of target tracking. Most of the improvement due to use of features in the data association is during the time period when one or more tracks are flying in very close proximity (measurement error larger than track separation). During other times, data association based on kinematic information alone is accurate and the use of features does not provide much added value.
3.2
A. Track mis-associations occur when a measurement is paired with a wrong track and is defined as Track misassociation = 100 ×
# wrong associations
%
(6)
# total associations
B. Track RMS position error is the root mean square kinematic position error between the ground truth (actual trajectory) and estimated track position and is defined as
RMS error =
1 N
( xitrue − xiestimated ) 2 + true estimated 2 ( y − y ) + ∑ i i i =1 ( z true − z estimated ) 2 i i
Data association with kinematic measurements only
N
(7)
Data association with kinematic and feature measurements
Table 1 (column labeled with features) shows the results based on use of both kinematic and feature measurements. Using (5) for data association results in excellent data association (only 2% incorrect trackmeasurement pairing). The feature likelihood score is fed to the joint likelihood computation module. The resultant scores form the 2-D assignment matrix. Once the associations are computed, the tracks are accordingly updated. As a consequence of correct associations, both the track quality and track ID estimates are very accurate. No false track ID was declared in this case. Table 1 Tracking performance without and with features Performance metric Track misassociations Track RMS position error Non-declared track ID False track ID
Here N is the total number of time samples and (x,y,z) are the Cartesian position coordinates. Track ID accuracy is the accuracy in declaring a correct target type of a track based on the PAS output. When data association is not correct, the attribute measurements will be associated with the wrong track and the ID estimation done accordingly will result in erroneous ID declaration. A track is declared to be a certain target type when the degree of support [6,7] exceeds a certain decision threshold. For this simple case here and for ease of explanation, we chose this threshold to be 0.8.
Without features 23%
With features
140m
62m
14%
0.6%
4%
0%
2%
This above simple example shows how feature data could be used to improve data association thus improving target ID estimation as well as data association accuracy
C. Non-declared track ID: If the degree of support of any target type for a track is below this threshold, we do not declare the ID (target type) for this track. The number of such non-declared track ID cases is used as a measure of track ID accuracy.
4
Conclusions
In this work, we have presented a tracker that uses feature data to improve data association thus improving both target kinematic state and ID estimation. Other more advanced methods based on the same idea include but are not limited to MHT, etc. and will further improve
D. False track ID: When the declared ID of a track is different than the actual ID (available from ground truth), we use that as a measure of false track ID.
15
ID estimation over that derived from a simple global nearest-neighbor data association tracker.
References [1] S. Blackman and R. Popoli, Design and Analysis of Modern Tracking Systems, Artech House, 1999. [2] R.W. Sittler, "An Optimal Data Association Problem in Surveillance Theory," IEEE Trans. On Military Electronics, Vol. MIL-8, April 1964, pp. 125139. [3] E. Bosse and J. Roy, "Fusion of Identity Declarations from Dissimilar Sources using DempsterShafer Theory", Optical Eng., Vol. 36, No. 3, March 1997, pp. 648-657. [4] E. Blasch and L. Hong, “Data association through fusion of target track and Identification Sets, 3rd international Conference on Data Fusion, Fusion 2000, July 2000. [5] K.J. Sullivan, M.B. Ressler, R.L. Williams, “Signature-aided tracking using HRR profiles”, Proc. SPIE Conf. On algorithms for synthetic aperture Radar Imagery VIII, vol. 4382, p. 132-142, 2001. [6] J. Kohlas and R. Haenni, “Assumption-Based Reasoning and Probabilistic Argumentation Systems”, Tech. report 96--07. Institute of Informatics, University of Fribourg. 1996. [7] B. Anrig, R. Bissig, R. Haenni, J. Kohlas, and N. Lehmann, “Probabilistic Argumentation Systems: Introduction to Assumption-Based Modeling with ABEL”, Institute of Informatics, University of Fribourg. 1998. [8] J. Pearl, Probabilistic Reasoning in Intelligent Systems: Network of Plausible Inference, Morgan Kaufmann, 1988. [9] S. Blackman, Multiple-Target tracking with Radar Applications, Artech House, 1986.
16
