Modelling Competitive Behaviours by a

0 downloads 0 Views 551KB Size Report
Modelling Competitive Behaviours by a hierarchical HMM architecture. Shih-Yang Chiao and Prof. ... paper is concerned with the modelling of the behaviour of.
Modelling Competitive Behaviours by a hierarchical HMM architecture Shih-Yang Chiao and Prof. Costas S. Xydeas Department of Communication Systems, Lancaster University, UK {s.chiao, c.xydeas}@lancaster.ac.uk

Abstract- A decision making mechanism based on Hidden Markov Models (HMMs) was presented in the paper. This paper is concerned with the modelling of the behaviour of players operating in a competitive environment that is characterized by interactions amongst players or groups of players. Thus a new, on-line, hierarchical, probabilistic modelling architecture with a probabilistic decision tree was developed for the purpose of on-line behaviour recognition that accepts HMM behaviour probabilities of player and effectively segments their behaviour-with-time trajectories. This allows the location of important points in time where behaviour changes occur. Furthermore, the hierarchical nature of the system allows individual player classification results to be used towards the modelling and classification of higher-level tactical behaviours of groups of players, as defined within an application envelope. The system is applied in a relatively simple 2-D “air patrol” scenario and system simulation performance results are provided in terms of certain useful metrics.

1. INTRODUCTION This paper addresses the issue of modelling the behaviour of players operating in competitive environments in order to make optional decisions for users. For the purpose of developing a general, on-line classification/decision methodology, a hierarchical network structure is proposed and used to model behaviours hierarchically. The network structure is based on Hidden Markov Models (HMM). HMM is a rigorous probabilistic classification/prediction framework that has been successfully applied in several applications, in general and speech recognition in particular [1][2][3]. Furthermore, its natural capability of dealing with time varying patterns of arbitrary lengths is attractive from a behaviour-modelling point of view, due to the expected variability in the time lengths of player behaviours. Using computer simulation experiments the performance of the proposed system is evaluated within the framework of simple 2D “air-patrol” flight scenarios. Flight scenarios were therefore simulated and participating aircrafts acted as players. The behaviour of individual players is classified on-line, in the lower layer of the proposed structure, which also includes a decision tree that effectively segments player behaviour-with-time trajectories into a concatenation in time of different behaviours. Furthermore the system employs the first level on-line classification results as input information

ISBN: 1-9025-6009-4 © 2003 PGNet

into the next HMM based layer whose mission is to classify higher-level tactical behaviours of groups of players. Section 2 of the paper provides a description of the proposed system. The input database used, first to design and then to test the system, is discussed in section 3. Section 4 presents experimental results whereas section 5 provides conclusions and closing remarks. 2. SYSTEM DESCRIPTION The proposed system accepts observation data vectors Ot, at discrete times t1, t2, … in relation to N observed players operating within its environment, see Figure 1. Following certain data processing operations, input player data is presented to corresponding Layer 1 HMM classifiers whose output provides the maximum likelihood probability that players performs one of a finite number of behaviours. An HMM models a sequence of observations (Yt) by specifying the probabilistic relationship between observations and a sequence of hidden states S in a Markov transition structure linking the hidden states. The model assumes that given St, Ot is independent to all other observations and states and that St is independent of S1, S2, …,St-2 , given St-1 , where

( ) St ∈ 1,2,..., p . Thus given a number of behaviours bhi, [ ] and λi = A, B, π i=1,2…n behaviour models, which are

calculated via an HMM training process, classification is performed on a per player basis by estimating the maximum likelihood probability P(Yt / λi ) [1].

Figure 1. System Block Diagram

The data processing block that accepts input sensor and other application related data, performs the processes of normalization and compression often associated with feature extraction and provides observation sequences to HMMs. It is important at this point to consider the on-line nature of the required classification and the fact that observation sequences represent a succession of behaviours bhi .This implies the need of a segmentation process to be applied across the “evolution with time” of observation sequences. Segmentation will provide the points in time where the behaviour of a player changes from bhi to bhj . An accurate segmentation process is of course important, in obtaining accurate HMM classification performance for the different behaviours, since it will allow only the appropriate part of the signal generated by a single behaviour to be presented to HMM for classification. Furthermore, the accurate location in time of changes in behaviour can be application critical information. Thus an integrated segmentation/classification procedure has been developed and is performed for each Level 1 HMM network, see Figure 2.

Figure 2. Integrated HMM classification and segmentation process.

3. EXPERIMENTAL DATA Experimental data was produced in support of system design and performance evaluation, for simple 2-D airpatrol applications [5][6]. In the first case three basic Level 1 behaviours where considered, namely: Line, CAP, Intercept and from these thirty-three experimental scenarios where produced. Furthermore, three Level 2 tactical behaviours were considered. These are (i) two patrol aircrafts perform CAP and once an opponent is detected within a specified distance, one of the aircrafts commences an intercept action, (ii) two pairs of aircraft perform CAP and once an opponent is detected within a specify distance an aircraft from the other CAP commences an intercept, and (iii) two

pairs of aircraft perform CAP and once a opponent is detected, another aircraft that is on the ground initiates take-off and intercept. An instance of the second tactical Level 2 behaviour is shown in Figure 3. Thus HMM system networks were designed for both the 2-D applications, during a training process involving data provided from 1000 different versions of each basic behaviour,. Furthermore, system performance evaluation was performed using data generated from 100 different versions of each scenario type.

4. SYSTEM PERFORMANCE EVALUATION As expected, errors in on-line behaviour classification occur at the very beginning of a given scenario, due to the limited number of time points and associated data that is available at the input of the system. Because of this, the system is allowed to “look ahead” i.e. to gather input data from the first 20 time points, before on-line classification is initiated. Errors also occur at points in time following changes in behaviour.

Figure 3. An example of a Level 2, type 2 scenario.

Two metrics were defined and used to evaluate the system’s average classification performance. Averaging is calculated over the 100 different trials performed for each scenario used in these experiments. The first metric called Response Time Measurement (RTM) is defined as: RTM =

1 N1

N1

∑ [CT ′(i) − CT (i)]

(1)

i =1

where CT and CT’ are the time indices of actual and estimated behaviour changes respectively and N1 is the total number of behaviour changes. RTM provides an average estimate of time points that the system takes to detect behaviour change, following an actual change. Thus the smaller RTM is the quicker the system identifies

accurately changes in the behaviour of player. second Error Measure (ERM) is defined as:

ERM =

1 T

The

T

∑ Dif (r( j),r′( j))

(2)

j =0

where T is the total number of time points in a scenario. Dif (a, b) is zero when a= b; otherwise it is one. r and

r ′ represent the actual and estimated behaviours.

5. CONCLUSIONS An on-line, system architecture/decision mechanism is proposed in this paper that allows the hierarchical modelling and classification of the behaviours of players operating in competitive environments. The methodology employs HMM networks and hierarchical behaviour modelling can be performed with as many tactical and strategic behaviour layers as required in a given application domain. This method also employs an integrated segmentation/classification process that offers effective on-line classification accompanied with decision tree. Thus of a two layer system has been implemented and its operation tested in the case of simple 2-D airpatrol scenarios. The system can be part of sophisticated hierarchical situation awareness/ risk assessment/prediction process that is designed for competitive environments. ACKNOWLEDGMENTS The authors kindly acknowledge the numerous discussions and views provided by M. Everett and C. Patchett on the subject. REFERENCE

Table 1. RTM and ERM results for 2-D application

Table 1 illustrates RTM and ERM results obtained in the 2-D application Notice that given the initial 20 points delay, the overall ERM figure expressed as % is less than 1%, whereas over all scenarios average RTM is less than 3.5 time points. The effect of using level 2 feedback within the level 1 segmentation decision-making process is also highlighted by allowing the system to operate: (i) with feedback but without the 20 time points initial look ahead delay and (ii) without feedback but with the 20 points delay. The performance evaluation presented in Table 1 show the capability of system accompanied with feedback process and without setting delay points. Note that Level 2 classification performance obtained for the 2D application is 100%.

1.

L. R. Rabiner, “A Tutorial on Hidden Markov Models and Selected Application in Speech Recognition”, Proc. of the IEEE, Vol. 77, No.2, pp. 257-285, 1989.

2.

M. A. Mohamed, P. Gader, “Generalized Hidden Markov Models-Part I: Theoretical Frameworks”, IEEE Trans. on Fuzzy system. Vol. 8, No.1, pp.67-81, 2000.

3.

Z. Ghahramani, “An Introduction to Hidden Markov Models and Baysian Networks”, International Journal of Pattern Recognition and Artificial Intelligence. Vol. 15 No. 1, pp.9-42, 2001.

4.

Y. Bengio, “Markovian Models for Sequential Data”, Neural Computing Surveys, vol.2, pp. 129-162, 1999.

5.

R. L. Shaw, “Fighter Combat: Tactics and Maneuvring”, Airlife Publishing Ltd, England, ISBN 0-87021-059-9, 1985.

6.

K. N. Ross, R. D. Chaney, “Hidden Markov Models for Threat Prediction Fusion”, Processings of SPIE Vol. 4051, pp.300-311, 2000.