A COMPUTATIONALLY-EFFICIENT COLLISION EARLY WARNING SYSTEM FOR VEHICLES, PEDESTRIANS, AND BICYCLISTS Daniel Greene, Juan Liu, Jim Reich, Yukio Hirokawa, Akio Shinagawa, Hayuru Ito, and Tatsuo Mikami ∗
Abstract We describe a computational architecture of a collision early warning system for vehicles and other principals. Early warnings allow drivers to make good judgments and to avoid emergency stopping or dangerous maneuvering. With many principals (vehicles, pedestrians, bicyclists, etc) coexisting in a dense intersection, it is difficult to predict even a few seconds in advance, since there are an enormous number of possible scenarios. It is a major challenge to manage computational resources and human resources so that only the more plausible collisions are tracked and of those, only the most critical collisions prompt warnings to drivers. In this paper, we propose a two-stage collision risk assessment process, including (1) a preliminary assessment via simple efficient geometric computations which throughly considers surrounding principals and identifies likely potential accidents, and (2) a specialized assessment which computes more accurate collision probabilities via sophisticated statistical inference. The whole process delivers an expected utility assessment to available user-interfaces, allowing the user interfaces make discriminating choices of when to warn drivers or other principals.
INTRODUCTION Improved sensing and communication technologies have been incorporated in various intelligent transportation systems to provide users with significantly better information about their surrounding environment and the road ahead. Vehicles are now equipped with better sensors, and pedestrians and bicyclists are carrying intelligent devices. All this information can be exploited to prevent accidents, but it is extremely important to warn principals (vehicles, bicyclists, pedestrians, etc) early, to give them time to react to warnings and to apply their own additional judgment before action is taken to avoid accidents. This is in contrast to immediate warning systems, which are important, but often result in emergency stopping or maneuvering and can be uncomfortable or even cause other accidents. This is joint work of the Palo Alto Research Center and Fujitsu Limited. During the course of this work, D. Greene, J. Liu and J. Reich had been with the Palo Alto Research Center; Y. Hirokawa, A. Shinagawa, H. Ito, and T. Mikami had been with Fujitsu Limited. The authors can be reached at (in order): [greene, juan.liu]@parc.com,
[email protected], [y.hirokawa, shina, hayuru]@jp.fujitsu.com, and
[email protected]. ∗
1
Sensing and Communication Reasoning Layer: Preliminary Assessment
Specialized Assessment
User Interface
Figure 1: Overview of the Architecture However, early warning poses significant computational difficulty, especially at a busy intersection where there are many principals, and anticipating ahead even a few seconds involves tracking an explosion of possible trajectories. We must take a stochastic approach—devoting computation to the likely collisions rather than all possible collisions. Even then, we will be able to process hundreds of likely collisions, and we must be more selective with warnings to humans. Unnecessary warnings may distract users and make the system unusable. In this paper, we propose a computationally-efficient Collision Early Warning System (CLEWS). The architecture is shown in Figure 1. It consists of three main components: (1) sensing and communication, for signal collection from the physical surrounding environment, (2) user interface, which prompts warnings to drivers, and (3) a reasoning layer, which is the main focus of this paper. We do not discuss in this paper basic sensing and communication, we assume that the principal’s reasoning layer receives sensing information about other nearby principals. The algorithms in reasoning layer are designed to tolerate moderately noisy sensor data. Likewise, we are not addressing the design of in-vehicle user interfaces. Our system would benefit from recent user interface research on low-distraction ways of issuing warnings, e.g. in the periphery of head-up displays or with directional sound. We have taken great care to make the output of the reasoning layer useful to a wide variety of user interfaces (e.g., audio, indash, heads-up, and roadside signage) by providing an expected utility assessment that includes information necessary to make critical decisions on if, when, and how to issue warnings. The actual decisions are customized to the user interfaces. A central aspect of our architecture is the division of the reasoning layer into two parts, which we will refer to as “preliminary assessment” and “specialized assessment.” This architecture is shown in Figure 1. Roughly speaking, the preliminary assessor does a rapid geometric identification of potential accidents, and starts specialized assessors to do more detailed statistical analyzes of potential accidents when necessary. This dual approach effectively solves three critical problems. First it reduces the overall computational complexity. The geometric approach of the preliminary assessor is highly optimized for covering large numbers of principals and scenarios. Secondly, it efficiently manages computational resources. By first computing preliminary assessments of a wide range of possible accidents, the reasoning layer is in an ideal position to allocate computational resources to the computationally more expensive specialized assessors. Finally, it addresses the model selection problem for risk assessment. In real-world traffic, a large percentage of principals follow predicable patterns; the preliminary assessor sorts through all these patterns and selects models for specialized assessors. Among the collision detection/avoidance literature, CLEWS is unique in its emphasis on providing advance warning in dense intersections. It includes techniques to deal with the com2
y
s D(t)
s=4
T(s) s=3 s=0
s=1
s=2
t
x
Figure 2: A dynamics and a trajectory that, when combined, describe a vehicle slowing as it makes a left turn.
putational complexity and long-term modeling of the principal’s behaviors. Other collision detection work has focused on less complex (e.g. rural) intersections [1, 2]. Often linear dynamics models are used [3, 4], which are suitable and efficient for short-term prediction, but are limited in modeling driver maneuvering behavior over the time scales necessary for early warning. In contrast, the two-stage risk assessment approach CLEWS takes is specifically designed to handle busy intersections with complex principal motion. This is the main differentiator. Another key aspect of our system is the concept of risk assessment. This concept has appeared in the EU’s PROTECTOR project [5], which produces a “conflict risk analysis.” However, the PROTECTOR system focuses simply on probability, rather than taking into account the benefit of issuing a particular kind of warning, and does not consider warning timing or adaptation to different types of UI. PRELIMINARY ASSESSOR Preliminary assessment uses a library of likely trajectories for principals and applies efficient geometric tests to determine if two principals might collide. We’ll first describe the geometry of preliminary assessment. Motion of Principals In our architecture the motion of a principal is decomposed into a “trajectory” in the x-y plane, and a “dynamics” along the trajectory. This is based on the observation that principals usually follow highly predictable trajectories, for example, the curves followed by right turning vehicles in an intersection are very similar, but their speed and acceleration along those trajectories can be highly variable. A “trajectory” T (s) is a vector valued function from an arc length parameter s to the physical plane x-y. For our prototype system, T is a piecewise function of line segments and arc segments. A “dynamics” D(t) is a function from time t to distance s. For our prototype system, D is a piecewise function of periods of steady speed (i.e. at + b) and accelerations or decelerations (i.e. at2 + bt + c). A motion is a composition of a trajectory and a dynamics: T (D(t)) (see Figure 2 for an example). We have chosen a simple set of geometric primitives to represent the motion of prinicipals—precise fidelity is not critical, since all of the computations described later will make allowance for the principals not precisely following trajectories and dynamics. Geometric Computation: Segmented Cones The central computational element of the preliminary assessor is the segmented cone. Segmented cones are “worst case” descriptions of where principals might travel as they follow 3
Figure 3: A Segmented Cone with its Bounding Boxes scenarios. They are generated from a library of scenarios associate with the local road geometry. For example, there might be scenarios for vehicles stopping at an intersection, turning right, changing lanes, etc., and scenarios for pedestrians crossing in crosswalks or jaywalking. A segmented cone is a region defined in three dimensional space (including the time dimension, i.e. (x, y, t)). It covers most of the likely future locations of a principal following a particular scenario. So for example, if the scenario is “vehicle makes a right turn” then a segmented cone for that scenario describes a region S ∈ R3 . If the vehicle does make a right turn, then with very high probability the vehicles future coordinates (x, y, t) are inside of S. As t grows larger, it is harder to predict where a principal will be located, so segmented cones will flare with increasing t giving them the inverted cone appearance shown in Figure 3. We refer to these as “segmented” cones because they are assembled from prismatoids joined together at a fixed set of t values: t0 , t1 , t2 , t3 , . . .. For each prismatoid there is a bounding box, and there is an “overall” bounding box for the entire segmented cone. These bounding boxes are shown in red and blue respectively in Figure 3. The bounding boxes are the basis of highly efficient cone intersection: bounding boxes can be compared first, to determine if more comparisons are necessary. The generation of cones is based on the context of the principal—this context suggests which scenarios are likely and cones are generated for all likely scenarios (sometimes more than one per principal, as we shall see shortly). The context is defined as a conjunction of trigger conditions, for example, a vehicle is near the center of a left turn lane, and within 40 meters of an intersection, so a left turning cone should be generated. (Trigger conditions can be based on such things as the type of the principal, the location and velocity of the principal, and the state of nearby traffic lights.) Once a cone is triggered, the cone itself is created by combining “a priori” information from the scenario with “current state” of the principal to project a cone forward in time. Figure 4a illustrates the information used in cone generation, and Figure 4b illustrates the cone generated. In this case “a priori” information includes a trajectory from the scenario (a left turn), and current state includes the principal’s location and velocity. In this way the scenario is customized to the principals, e.g. a faster moving vehicle will generate a more extended cone. Computation of Preliminary Assessment Because segmented cone intersection is so efficient it can be repeated frequently (e.g. 2–5 times per second). In our prototype system, a vehicle repeatedly triggers cones for itself and any 4
Y 30
20
Diplacement of vehicle from trajectory.
40 10
30
Segmented cone is centered on the displaced trajectory.
Trigger conditions: Vehicle must be near trajectory. Velocity must be nearly parallel to trajectory.
X
Speed of vehicle determines initial slope of segmented cone into the T dimension. Scenario determines full shape of cone.
20 0 0
10
10 20 30
(a)
0 40
(b)
Figure 4: Cone generation: (a) trigger conditions and parameters, (b) generated cone.
Figure 5: Cone scenarios: (a) multiple cones (for stopping, proceeding straight, and turning left or right)
are triggered as a vehicle approaches an intersection, (b) two vehicles cones intersect; they might collide; a preliminary assessment is computed.
other principals being tracked. The generated cones are compared with each other to determine if they intersect. When two segmented cones intersect (e.g. figure 5b), this indicates a possible collision, and a preliminary assessment of that collision is computed. Because this need for preliminary assessment is based on large “worst case” sized cones, this potential collision may still be extremely unlikely, so we will choose some scenarios for a full “specialized” assessment that will arrive at probabilities and expected utilities that can help determine if warnings are warranted. The preliminary assessment will enable the system to allocate resources to those scenarios most deserving specialized assessment. Figure 6 shows in a stylized manner how a preliminary assessment is computed from a segmented cone intersection. The time until collision is assumed (conservatively) to be the point where the cones first intersect. The probability of collision is computed by comparing tiers level by level, estimating the probability based on the largest amount of overlap. We also estimate seriousness based on the kinetic energy and vulnerability of the principals. Once the preliminary assessment is completed, the product of the probability and the seriousness gives a utility estimate to rank the potential accidents for specialized assessment and possible warning. Managing Limited Computational Resources Preliminary assessments provide a basis for resource allocation for specialized assessment. There will be multiple approaches to specialized assessment (several are described in the next section) that will provide a trade-off between amount of computation used and the accuracy of results achieved. Using the results of preliminary assessment, we can determine which scenar5
ented Cone Other Segm
Co ne yS eg me nte d Pr im ar
Time
Primary
Probability is estimated as a measure of the overlap of the two cones Time-to-accident is estimated (conservatively) as time when the first overlap of the two cones occurs. Time-to-warn allows for a safe deceleration before time-to-accident..
Other
Figure 6: Deriving a preliminary assessment from the segmented cone intersection ios deserve specialized assessment, of those deserving specialized assessment, which algorithm should be used. These choices have several objectives: 1) to avoid unnecessary warnings because the preliminary assessor has overestimated the collision, 2) to avoid overlooking scenarios because the preliminary assessor has underestimated a potential collisions, and 3) when warnings are presented to the driver, the timing of the warning (see the UI section) will benefit from careful specialized assessment. We have explored several cost-benefit approaches to allocating resources for specialized assessment. Because there will be enough computational resources to perform several orders of magnitude more specialized assessments than will prove serious enough for warning, we believe that the the following simple and robust approach will work well: we use a time slice between 0.2 and 0.5 seconds. At the beginning of the time slice, roughly 10% of the time is allocated to a thorough preliminary assessment of all triggered scenarios for all nearby principals. A next portion of the time slice (e.g. 60%) is allocated to speciallized assessment using a moderately expensive and reasonable accurate IMM based on Kalman filtering (described later in the next section). This kind of tracking is fast enough that many specialized assessments can be run during this portion of the time slice; this will almost certainly cover the most critical scenarios. Finally the remainder of the time slice (e.g. 30%) is devoted to more accurate specialized assessment algorithms. This fixed division of the time slice is not precisely optimal from the cost-benefit approach, but has good cost-benefit characteristics, and it is more robust because it always allocates a fixed amount to (60%) to obtain thorough coverage of the more critical scenarios. Finally we note that not all preliminary assessments require specialized assessment. Our prototype system supports “erratic” behavior patterns, where the trajectory of the principal is not very predictable, and so heavily flared cones are generated to reflect the lack of good “a priori” information. These erratic scenarios would include such behaviors as a driver turning the wrong direction on a one-way street, or a driver falling asleep and drifting into oncoming traffic. The “erratic” scenarios do not have good models for specialized assessment, yet the need for warning is very high, so the preliminary assessment alone will be sufficient to provide warnings to the driver. SPECIALIZED ASSESSOR To enable early warning, the specialized assessor needs to predict collision well into the future. We formulate the risk assessment problem as a statistical inference problem based on relatively long-term (a few seconds, as opposed to a fraction of a second) models. This sets us apart from most existing collision warning systems such as [3, 6, 2, 7, 8] which focus on providing immediate collision warnings based on short observation data. Our specialized assessor 6
(a)
(b)
Figure 7: Models: (a) slotcar, the straight red line is the desired trajectory, the blue curve is the actual
motion path, (b) driver alertness as a Markovian process.
is model-based: principals are associated with a library of models, such as a dynamics model specifying the way the vehicle moves, and the driver’s mental model specifying how the driver acts based on the surrounding environment. With these models, the specialized assessor fits the data and make inference about the underlying state (e.g., position, velocity, whether the driver is paying attention). Based on the inferred state, the specialized assessor makes predictions into the future and generates a risk assessment which can be used to make informed decisions about whether a warning is necessary. Model-based approaches are flexible, allowing new models to be integrated and existing models to be improved without the need to completely re-design the system. Models The dynamics model specifies the way a principal moves. In our prototype, we use a “relaxed slotcar” model, shown in Figure 7a. A slotcar is a powered miniature toy automobile which is guided by a groove or slot in a track. In our prototype model, the principal follows a pre-defined desired trajectory (the red straight line in the figure), but is allowed to deviate from the trajectory to simulate real-world motion. The principal is subject to two “model” forces: a perpendicular force pulling it towards the desired trajectory if it deviates, and a tangential force adjusting the speed toward some target speed. The target speed is a parameter which may be dependent on several factors: the driving habit of the driver, the speed limit at the current location, and the speed of cars around the principal. Driver (and/or pedestrian and bicyclist) alertness is also important to risk assessment. If a driver is alert, he/she has less chance colliding with others, even if the vehicle is currently going fast. The driver alertness is formulated as binary: 0 for non-alert, and 1 for alert. The alertness model is described as Markovian (see Figure 7b): if the driver is alert at time t, he has a high probability for staying alert at the next time instant t + 1. Same is true for the transition from non-alert back to alert. The parameters α and β characterizes the drift and are assumed to be small (< 0.1). Furthermore, the transition is asymmetric and dependent on the surrounding environment. For example, if the driver is alert and realizes the potential danger of hitting another vehicle/pedestrian, thee he will pay conscious attention and will not drift to non-alert state before the danger clears. With this alertness model, the specialized assessor seeks to infer about the driver’s alertness from the vehicle history data.
7
Risk Assessment Using Statistical Inference With the models described above, the specialized assessor assess risk by inference in the state space (pos, vel, alertness). Two questions need to be answered at any time t. First is the filtering (state estimation) problem: given the observation history data z t up to time t, what state the principal is in. The second is the prediction problem: given the principal’s current state at t, what state the principal will be in at time t + 1, t + 2, · · · , t + N , where N is the prediction horizon. For the filtering problem, we refer to the generic formulation of sequential Bayesian filtering. Sequential Bayesian filtering updates the posterior of underlying state x t+1 as: Z t+1 t+1 t+1 t+1 pd (xt+1 |xt ) · p(xt |z t ) dxt . (1) p(x |z ) ∝ ·po (z |x ) · X
The integral makes a single step of prediction bringing the previous state up the the current time and then applying a new external measurement to modify the state. It does this filtering using an object dynamics model pd (xt+1 |xt ). In our prototype, the dynamics obey the relaxed slotcar model and the alertness model. The prediction is then multiplied by a likelihood, reflecting the contribution of observation z t , using an observation model po (z t+1 |xt+1 ). The filter (1) is sequential: the current filter distribution p(xt+1 |z t+1 ) is computed from the previous filter distribution p(xt |z t ) at every step. The prediction problem is simple: it is just the integral in (1). For the filtering and prediction problem, we have experimented with several types of techniques. The first type is the well-known particle filter [9], which represents a continuous probability density function as a set of weighted Monte-Carlo samples, referred to as a particle set. Each particle evolves its state independently based on the dynamics models and adjusts its weight based on observation data. The advantage of particle filter is its remarkable flexibility. It can accommodate almost any form of dynamics pd (xt+1 |xt ) and observation model po (z t+1 |xt+1 ). The drawback is its high computational complexity: a large number of particles need to be simulated to maintain the representativeness of the particle set. In practical situations, the high computational complexity may make this infeasible. We reserve particle filtering for only the most serious potential collisions and the most complex models. The second type of approach is the commonly-used Kalman filter[10, 11]. It is a special case of sequential Bayesian filtering under the assumption that the object dynamics and the observation model are both linear and Gaussian. The Kalman filter updates the mean and the covariance of the belief p(xt |z t ) recursively as measurements are observed via simple Kalman Gain linear operations. It is computationally efficient, but the performance is often limited by its modeling assumptions. In the collision warning problem, we linearize principal state (pos, vel) with respect to the principal trajectory to comply with the linear assumption. However, the driver alertness model is non-linear and bi-state (which is clearly non-Gaussian). To tackle this difficulty, we use a method known as the Interacting Multiple Model (IMM) algorithm [12, 13], which is specifically designed for filtering in linear systems with Markovian transitions. IMM assumes multiple modes of operation. For example, a vehicle may have multiple maneuvering modes (stopping, accelerating, steering, etc). Within each mode, the principal follows a linear dynamics model. The principal may switch from one mode to another, and the switching mode event is assumed Markovian. IMM formulates the filtering problem as a hypotheses management problem, where the hypotheses {H 1 , H2 , · · · , HKP } are the K different maneuvering modes, with probability {µ1 , µ2 , · · · , µK } respectively, and i µi = 1. The goal 8
Benefit of Issuing Warning
knee seriousness
collision time
Time of Warning (future t’ relative to current time)
Figure 8: An example benefit function. is to update the posterior probability for each hypothesis µti based on the observation history z t . The question is, can we estimate µt sequentially as time t advances? The answer is yes. The technical details of IMM are described in [13]. The intuition is as follows: at any point of time t, we start with a hypothesis prior set {µtj , j = 1, · · · , K}, where µtj = p(Hjt |z t ). Now assume at time t + 1, hypothesis Hi is true. Given that the system is linear and Gaussian under Hi , the Kalman filter is appropriate. One can update the state estimate and corresponding covariance, and compute the likelihood p(z t+1 |Hi ). The hypothesis probability can then be updated as: X µt+1 ∝ µtj · p(Hit+1 |Hjt ) · p(z t+1 |Hi ) i j
We think IMM are particularly suitable for collision prediction applications because they handle the two-state (alert vs. non-alert) model above very easily, and we expect that specialized tracking of most real-world driving will contain several distinct modes like these. The computation cost is K-times the normal Kalman filter, but it is still much more efficient than the particle filter. Computation of Specialized Assessment Based on the forward prediction described in earlier sections, a specialized assessor computes an assessment for every time slice. The assessment includes the following information: 1) documentation of scenarios, this includes trajectories, text descriptions, and other useful information for presenting warnings, 2) P (t), the probability assessed now (time t) that there will be a collision in the future assuming that the principals follow the scenarios. 3) T (t), the expected time of the collision, 4) L(t), the predicted location of the collision, 5) B(t, t 0 ), the predicted benefit of warning assessed now (time t) but estimating the expected benefit of warning at future time t0 relative to the current time. This benefit function, illustrated in Figure 8, will be used to plan warnings—delaying warnings to increase certainty, when there is no loss of benefit. The prominent feature of the benefit function is a “knee” in the function. Before the knee the driver has time to consider the warning, react, and safely avoid the accident (e.g. slow the vehicle). The height of the benefit function before the knee is proportional to expected consequence of the warning in the seriousness of the accident that would be avoided. ASSESSMENTS BECOME WARNINGS: THE USER INTERFACE We do not address UI design, but in this section we describe how to connect the assessments with user interfaces. The ultimate goal of an early warning system is to issue warnings to the driver in a way that prevents accidents and avoids dangerous emergency maneuvers. The decision to warn will be dependent on the characteristics of specific UIs. Warnings displayed on 9
Red – Supplied by Specialized Assessor Blue – Customized for UI
(1 – P) collision will not occur
P collision will occur
distraction penalty - D (1-Q) warning ignored
distraction penalty - D
Q warning heeded
B(0) benefit of warning
Figure 9: Refining the assessment P B(0) according to the Q and D characteristics of the UI. head-up displays will have different characteristics that warnings issued with sounds. Therefore we include in the architecture “decision adaptors,” user interface specific software that receives all the assessments and makes the critical warning decisions. While it would be possible for each UI to provide its own custom decision adaptor, in this paper we describe a basic decision adaptor that can be customized with a few parameters specific to the UI; it then filters all the assessments, and passes on to the UI only those assessments that require immediate warning. While this general-purpose adaptor is a good starting point, it does not cover all the possibilities that could be build into a customized decision adaptor, such as aggregation of warnings based on location. A Basic Decision Adaptor For this decision adaptor, the UI is characterized by 5 parameters: 1)warning rate W , a comfortable rate at which warnings can be absorbed by the driver, 2)maximum warning rate M , the maximum rate at which warnings will be issued in a crisis, 3) efficiency of warning Q, a factor between 0 and 1 the describes the effectiveness of warnings communicating to the driver, 4) rate at which the same warning will be repeated R, in case the driver has not headed the warning, 5) penalty for an unnecessary warning, D, and 6) threshold for warning, T . These parameters form a simple UI model that the decision adaptor uses to customize the warnings to the UI. Fundamentally it is best to warn the driver about the most serious potential accidents (i.e. highest expected utility for warning). The expected utility is roughly the probability of collision times the benefit of the warning, the height of the curve in Figure 8. However, in the basic decision adaptor, the parameters of the UI model can be used to customize the utility measure according to a more “pragmatic” ranking formula: R = P QB(0) − (1 − P Q)D
(2)
which adds a degree of realism diagrammed in Figure 9 (the ideal is Q = 1 and D = 0). We have chosen as simple a formula as possible that still captures important characteristics for the real utility of warning. However, this ranking alone is not sufficient to time warnings. Even when the system is predicting a serious collision, it may not be best to warn the driver immediately. When there is still plenty of time, waiting may resolve the problem: drivers can react on their own or improved accuracy of assessment can eliminate the need for a warning. Waiting to avoid warnings will 10
reduce distracting warnings and avoid frustrating drivers enough that they disable the warning system, problems that can cause additional accidents. Fortunately, the benefit function in Figure 8 includes a “knee” which is the critical last opportunity to warn the driver with enough time to avoid a collision. Warnings can wait until their “knees,” unless there are multiple serious collisions that require spacing out warnings, to give the driver time to absorb each warning and form judgments about appropriate actions. (The basic decision adaptor assumes the UI should avoid issuing simultaneous warnings; UIs supporting simultaneous warnings, such as heads up displays, can customize their decision adaptor, or use very high rates W in the basic decision adaptor.) The basic decision adaptor attempts to schedule all pending warnings before their respective knees, using a warning rate less than W . This leaves extra space, so that unforeseen warnings can be added, if necessary, using a higher warning rate M . The basic adaptor treats knees in the pending assessments as deadlines, and schedules the lowest ranked assessments first in the backwards direction— thereby favoring higher ranked assessments in the forward direction. After scheduling, if there is time remaining before the first warning in the schedule, then no warning is issued in the current time slice and the procedure is repeated again in the next time slice. However, if the scheduler cannot complete a schedule with time to spare, then a warning is necessary and is issued from the beginning of the schedule. It may be necessary to recompute the schedule with spacing M to accommodate all the warnings. In extreme cases the scheduler may issue a “overload” warning indicating that it cannot present all the warnings. BENCHMARKING OF COMPUTATIONAL COMPLEXITY The two-stage CLEWS architecture has been prototyped in Java. This is a early, high-level language implementation that lacks tuning and performance optimization. We report these results to demonstrate good computational performance although we expect significant improvement would still be possible. Our performance results are measured using a single processor Pentium-4 CPU running at 2.8 GHz. For the preliminary assessor, the key computational element is the segmented cone creation and intersection. For a 15-tier cone, creation takes approximately 70 µsec (microseconds), and intersecting two cones takes between 0.1–9 µsec, depending on the overlap. This is extremely efficient—the preliminary assessor can handle thousands of scenarios simultaneously. The specialized assessor is more computation-intensive. For this we have benchmarked the IMM Kalman filter. A tracking step takes around 3–7 msec (milliseconds). This means a processor dedicated to specialized assessment can handle 140–300 tracking updates per second. Combining the benchmarking results above, the collision early warning architecture shows the potential to analyze complex intersections (with a few hundreds principals) and warn drivers of the most serious collisions, using only a modest amount of computation power. CONCLUSION We have described the computational architecture for a collision early warning system which uses a two-stage reasoning layer. The reasoning layer lies between sensor data and the user interface. It applies a library of scenarios, first using highly efficient geometric comparisons, and then selectively applying more elaborate statistical analysis to compute an assessment of potential accidents that contains enough information for the user interface to effectively decide if, when, and how to warn the driver. This two-stage assessment is computationally efficient 11
and can work in real-time for complex intersections. Acknowledgments We would like to thank Fujitsu Limited for sponsoring this research, and in particular Takashi Aoki, Yoshihiro Aragane, Kazuo Asakawa, and Tomoyoshi Takebayashi for their foresight and direction in formulating this joint project. We would also like to thank PARC for support, in particular, Teresa Lunt for her guidance, and our intern student Deepan Thavarajah for his contribution to implementation. References [1] K. Funderburg, “Update on intelligent vehicles and intersections,” 2004. Available at http:// tinyurl.com/y4c26v. [2] J. Misener, R. Sengupta, and H. Krishnan, “Cooperative collision warning: Enabling crash avoidance with wireless technology,” in Proceedings of the ITS World Congress, 2005. [3] O. Aycard, A. Spalanzani, M. Yguel, J. Burlet, N. D. Lac, A. D. L. Fortelle, T. Fraichard, H. Ghorayeb, M. Kais, C. Laugier, C. Laurgeau, G. Michel, D. Raulo, and B. Steux, “Puvame - new french approach for vulnerable road users safety,” in Proceedings of the IEEE Intelligent Vehicles Symposium, 2006. [4] S. Atev, O. Masoud, R. Janardan, and N. Papanikolopoulos, “Real-time collision warning and avoidance at intersections,” Tech. Rep. MN/RC 2004-45, University of Minnesota, 2004. Available at http://www.lrrb.org/PDF/200445.pdf. [5] G. D. Nicolao, A. Ferrara, and L. Giacomini, “A collision risk assessment approach as a basis for the on-board warning generation in cars,” in Proceedings of the IEEE Intelligent Vehicle Symposium, pp. 436–441, June 2002. [6] D. M. Gavrila, J. Giebel, and S. Munder, “Vision-based pedestrian detection: The protector system,” in IEEE Intelligent Vehicle Symposium (IV), (Parma, Italy), June 2004. [7] K. Stubbs, A. H, O. Masoud, C. MicMillen, H. Veeraraghavan, R. Janardan, and N. Papanikolopoulos, “A real-time collision warning system for intersections,” in Intelligent Transportation Systems America, May 2003. [8] H. Veeraraghavan, O. Masoud, and N. Papanikolopoulos, “Computer vision algorithms for intersection monitoring,” IEEE Transactions on Intelligent Transportation Systems, vol. 4, no. 2, pp. 78–89, 2003. [9] A. Doucet, N. de Freitas, and N. Gordon, Sequential Monte Carlo methods in practice. SpringerVerlag, New York, 2001. [10] R. E. Kalman, “A new approach to linear filtering and prediction problems,” Transactions of the ASME - Journal of Basic Engineering, vol. 82, pp. 35–45, 1960. [11] H. Sorenson, Kalman Filtering: Theory and Application. IEEE Press, 1985. [12] H. Blom and Y. Bar-Shalom, “The interacting multiple model algorithm for systems with markovian switching coefficients,” IEEE Trans. Automatic Control, vol. 33, no. 8, pp. 780–783, 1988. [13] M. K. Kalandros, L. Trailovic, L. Pao, and Y. Bar-Shalom, “Tutorial onf multisensor management and fusion algorithms for target tracking,” in Proc. Amer. Control Conf., (Boston, MA), July 2004.
12
