Application of Artificial Neural Network Models to ... - Semantic Scholar

2 downloads 0 Views 151KB Size Report
Nov 15, 2002 - California State University, Sacramento. 6000 J Street, Sacramento ..... Activity Analysis: State-of-the-Art and Future Directions, in ... 2000, Eindhoven, The Netherlands: The European Institute of Retailing and Services Studies.
Application of Artificial Neural Network Models to Activity Scheduling Time Horizon Sean T. Doherty Department of Geography & Environmental Studies Wilfrid Laurier University Waterloo, Ontario Canada N2L 3C5 Tel : 519-884-0710, ext. 2044 Fax: 519-725-1342 Email: [email protected] and Abolfazl Mohammadian Department of Civil Engineering California State University, Sacramento 6000 J Street, Sacramento California, 95819-6029 Tel: 916-278-5990 Fax: 916-278-7957 Email: [email protected]

Nov 15, 2002

Word Count: 7253

Paper submitted to the 82nd Annual Meeting of the Transportation Research Board Washington, DC, January 12-16, 2003.

TRB 2003 Annual Meeting CD-ROM

Paper revised from original submittal.

Doherty and Mohammadian

2

Abstract. Over the past few years, machine-learning methods have expanded enormously. These approaches are increasingly being applied to areas of exploratory data analysis, prediction, and classification problems. At the same time new analytical techniques are expanding, new conceptual approaches to modelling travel are emerging in an effort to improve travel demand forecasting and better assess the impacts of emerging transportation. In particular, the shift towards activity-based travel analysis has led to the development of activity scheduling models. One of the key features of emerging models of this type is the attempt to simulate the order in which activities are added during a continuous process of schedule construction. In practice, a fixed order by activity type is often assumed – for example, work activities planned first, followed by more discretionary activity types. Using observed data on the scheduling process from a small sample of household from Quebec City (Canada), a neural network model is developed in this paper that classifies activities according to the order in which they were planned - or “planning time horizon” (preplanned, planned, or impulsive). A variety of explanatory variables were used in the model related to individual, household, and activity-based characteristics such as spatial and temporal fixity. The model developed exhibited a relatively high degree of prediction on test data, especially for the extreme categories of planning time horizon (preplanned and impulsive). These results suggest that machine-learning algorithms could be used to predict the order in which activities are selected in emerging activity scheduling process models, thereby avoiding static assumptions related to activity type.

TRB 2003 Annual Meeting CD-ROM

Paper revised from original submittal.

Doherty and Mohammadian

3

INTRODUCTION Over the past few years, machine-learning techniques have expanded enormously. These approaches are increasingly being applied to problems formerly reserved for formal statistical approaches, particularly in the areas of exploratory data analysis (data mining), prediction, and classification problems. Part of the reason for this has to do with research trends but there are also some potential advantages associated with such techniques, including the ability to model non-linear systems , the ease with which symbolic, nominal or categorical variables can be included, and the ability of these methods to deal with noisy data (difficult to fit into a statistical model). Case-based learning (k-nearest neighbor), decision trees, neural networks, genetic algorithms, and other simple generally applicable learning techniques often make it easier to find interesting connections among the variables in databases. While use of traditional techniques could take days or months to find an optimal model structure for a large database, a machine learning algorithm like a neural network or a genetic algorithm could find the answer automatically in a much shorter time, sometimes even in minutes or a couple of hours (1). At the same time new analytical techniques are expanding, new conceptual approaches to modelling travel are emerging in an effort to improve travel demand forecasting and better assess the impacts of emerging transportation. In particular, the shift towards activity-based travel analysis has led the development of models of “activity schedules”. Activity schedules for a day (or longer), includes the interdependent choices of what activities to participate in, where, at what time (including choices of start and end times), coupled with mode and route choices. In order to support these models, travel behavior researchers have been increasingly calling for more in-depth research into the underlying activity scheduling process (2-5). This is especially important since the (re)scheduling process is viewed at the core of many of the changes in travel behavior brought on by recent policy initiatives related to information technology and transportation demand management. The “scheduling process” can be defined as the planning, execution and adaptation of activity-related decisions over time within a household that underlies observed activity schedules. Developing models of activity scheduling has been met with considerable challenge. Part of the problem is that data collection and modelling efforts have focused largely on observed activity schedules rather than underlying decision processes. Two basic approaches have been adopted. The first approach focuses on the simultaneous choice among a set of activity-travel patterns, normally under a utility maximization framework (6-8). From a behavioral standpoint, these models have been criticized for assuming that the scheduling process is essentially static, leaving the process resulting in observed patterns largely unspecified. In reality, the scheduling process has been found to consist of a much more dynamic series of preplanning, revision, impulsive and opportunistic decision making leading up to and during execution of the schedule (9, 10). This represents a significant challenge to the assumptions that activities are planned and optimized in one step. A second approach to activity schedule modelling focuses more exp licitly on the replication of the sequencing of decisions made during the scheduling process. Some of the first attempts focused on the sequential choice of activities and locations (11, 12). Building on this approach, Kitamura et al. (13) proposed a simulation model in which the next activity type to be engaged in

TRB 2003 Annual Meeting CD-ROM

Paper revised from original submittal.

Doherty and Mohammadian

4

is first chosen, followed by it’s duration, then location, assuming that activities are planned and executed in sequence. A series of additional sequential models have evolved from the SCHEDULER conceptual framework as proposed by Gärling et al. (14) including SMASH (15)and models by Gärling et al. (16) and Kwan (17). The SCHEDULER framework proposes that a set of activities is chosen from the individual’s memory and a schedule is formed by selecting activities with the highest priority followed by attempts to find less prioritized activities that fit into open time slots. Some of the most recently proposed sequential models include Albatross (18), and the recently operationalized TASHA (Toronto Area Scheduling Model with Household Agents) model (19). In Albatross, a sequencing rule is proposed that stipulates that mandatory activities be completed before discretionary ones, and out of-home before in-home activities. The TASHA model assumes that the process of generating a person’s schedule starts with the selection of activity type in a fixed order (work, joint other, joint shopping, individual other; individual shopping), followed by the choice of start time, duration, and location. Travel is then added between activities with different locations. One of the overriding themes in these models is the notion that activities are scheduled/planned in some form of order – or in other words, scheduled on varying time horizons ranging from planned in advance to impulsive. However, in practice, simplifying assumptions are most often adopted in the specification of planning time horizon - for example, by ordering mandatory activities first, followed by discretionary ones, or by some other static method. The validity of this assumption is of some concern, since the planning of certain activities for inclusion (or modifications, deletion) during the scheduling process is clearly not always fixed – certain circumstances will invariably arise wherein seemingly higher/lower priority activities are chosen. These planning time horizon are also certainly not the same for all people, especially in an era of changing work-life balance and emergence of “work” activities that are much more flexible than in the past as a result of trends such as telework. The development of a model or rule for the scheduling time horizon of activities that is dependent upon the nature of the activity (not simply the activity type) and the situation at hand would go a long way towards avoiding such restrictive assumptions, and making the model more amendable to a variety of people and situations. OBJECTIVES It is considerably timely that at the same time travel behavior modelers are struggling to replicate human decisions making processes in more depth with fewer assumption, applicable techniques for dealing with such decision systems are also beginning to emerge for practical use. In this context, the objective of this paper is to describe an application of machine learning techniques to an activity scheduling classification problem. More specifically, the goal is to develop a neural network model that classifies activities according to the time horizon to which they were planned - preplanned or impulsive – using explanatory variables related to individual, household, and activity-based characteristics such as spatial and temporal fixity. Such a model could be used to predict the order in which activities are selected in emerging activity scheduling process models, thereby avoiding static assumptions related purely to activity type.

TRB 2003 Annual Meeting CD-ROM

Paper revised from original submittal.

Doherty and Mohammadian

5

MACHINE LEARNING METHODS Machine learning methods deal with computer programs that can learn from experience. They can build systems that learn to recognize patters and discover relations among variables. A complete discussion of different machine learning methods is not possible within the scope of this paper. Therefore, only a brief description of two most commonly used methods; Artificial Neural Networks (ANN) and Genetic Algorithms (GA) which were used in this study are provided in this paper. Artificial Neural Networks (ANN) are a class of learning algorithms which consists of multiple nodes that communicate through their connecting synapses. What makes ANN different from discrete choice methods is its use of pattern association and error correction as the underlying mechanisms to represent a problem in contrast to the random utility maximization rule (20). Typically, a neural network consists of a set of nodes: input nodes receive the input signals, output nodes give the output signals, and a potentially unlimited number of intermediate layers contain the intermediate nodes. Complete explanation of theoretical aspects of neural networks can be found elsewhere (21-23). There are various different types and architectures for neural networks, but most can be classified as belonging to one of several major paradigms. Each paradigm utilizes different learning strategies to perform tasks and will have advantages and disadvantages depending on the user’s particular application. Some of the most commonly used neural network forms are Perceptrons, Back propagation networks, and Kohonen self-organizing map. Perceptron is a simple forerunner of modern neural networks, without hidden layers. Back propagation networks refer to a special type of neural networks that makes use of a hidden layer. A back propagation network starts with random weightings on its synapses. During the training of the network it is exposed to input values together with the correct output values. The network’s weights are adjusted until its responses are more or less accurate. Kohonen self-organizing map automatically maps an external signal space into a system’s internal representational space. Selforganizing maps are very useful for the performance of classification tasks. After an extensive search through different algorithms and networks, a Multilayer perceptron neural network is utilized in this study. Multilayer perceptrons (MLP) are layered feedforward networks and are explained in more detail in the following section. Other paradigms of neural networks such as Time Delay Neural Networks (TDNN), Continuous Adaptive Time (CATNN) networks, Probabilis tic (PNN) networks, Generalized Regression (GRNN) networks, Principal component analysis (PCA) networks, Radial basis function (RBF) networks, and Co-Active Neuro-Fuzzy Inference Systems (CANFIS) are out of the scope of this study. Interested readers are encouraged to see Principe and Euliano (22) for a more detailed explanation and examples of the above-mentioned networks. Genetic Algorithm (GA) is a class of machine-learning algorithm that is based on the theory of evolution. A collection of potential solutions for a problem compete with each other and the best solutions are selected and combined with each other according to a kind of ‘survival of the fittest’ strategy. In every generation, a new set of artificial creatures is created using bits and pieces of the fittest of the old generations. Genetic algorithms are not simple random walks; they

TRB 2003 Annual Meeting CD-ROM

Paper revised from original submittal.

Doherty and Mohammadian

6

efficiently exploit historical information to speculate on new search points with expected improved performance (24). Genetic algorithms are theoretically and empirically proven to provide robust search in complex spaces. These algorithms are computationally simple yet powerful in their search for improvement. MULTILAYER PERCEPTRON NEURAL NETWORKS Perceptron is a simple neural network, without hidden layers. A Multilayer perceptron (MLP) differs from the simple perceptron in two ways. First, it has additional layers of neurons between the input and output layers, known as hidden layers in order to improve the learning power of the network. Second, it uses a transfer function to modify the input to a neuron. Multilayer perceptrons (MLP) are layered feedforward networks typically trained with static back propagation. These networks are called multilayer and feedforward since they have multiple layers and data flow in one direction in the network. They have found their way into countless applications requiring static pattern classification. Their main advantage is that they are easy to use, and that they can approximate any input/output map. The key disadvantages are that they train slowly, and require lots of training data. The configuration of a typical MLP is shown in Figure 1. Neural networks are non-linear learning machines built from many Processing Elements (PE). Each PE receives connections from other PEs. The signals flowing on the connections are scaled by adjustable parameters called weights (wi). The PEs sum all these inputs and produce an output that is a non-linear function of the sum. This output becomes either a system output or is sent to the same or other PEs. Figure 1 also presents the structure of a processing element. A processing element is simply a sum of products followed by a threshold non-linearity (transfer function). Its input-output equation can be expressed as: N

Y = f ( x) = f (∑W i Z i + β )

[1]

i =1

where N is the number of input synapses to the processing element, Zi are the inputs, Wi are the weights, and β is a bias term. The transfer function f is a non-linear function, usually a smooth sigmoid shape function. The most common transfer functions are step, logistic, and hyperbolic tangent (tanh) functions. These three typical threshold functions are shown in Figure 2. step hyperbolic tangent logistic

 1 for x ≥ 0 f ( x) =  − 1 for x < 0 f (x) = tanh (αx) 1 f (x) = 1 + exp(−αx)

[2] [3] [4]

α is a slope factor and usually is set to 1.

TRB 2003 Annual Meeting CD-ROM

Paper revised from original submittal.

Doherty and Mohammadian

7

The major difference between the logistic and tanh functions is the range of their output values. The tanh function is an antisymmetric function (y intercept is zero), while the logistic function is not (y intercept is 0.5). The logistic function produces values between [0,1], while the hyperbolic tangent produces values between [-1,1]. The tanh function and logistic function are normally interchangeable, with the final selection determined by the desired range of the output (either [1,1] or [0,1]). Neural networks that use either of these two functions can be made equivalent by changing weights and biases that make a linear transformation in the input and output spaces. An artificial neural network is made up of many of these simple processing elements, highly interconnected and generally organized by layers with each neuron in a given layer connected to every neuron in the succeeding layer. Signals travel through these connections from neuron to neuron and finally generate an output. Some nets are capable of learning by themselves (unsupervised learning). The neurons in these types of nets continually monitor their inputs and keep adjusting their weights trying to mirror the signals they receive. Unsupervised learning nets are capable of modelling the probability distribution function of the input data. Other nets need an executor (supervised learning) to tell them whether they are correct in their responses, and, if not, by how much they are wrong. The difference between the expected response and the actual response given by a processing element is used, as feedback, to readjust the weights of the connections and further to reduce the error until the processing element eventually generates the correct response. This is a gradual process which usually takes many iterations to accomplish. By using this algorithm, there is a significant chance that over training will take place. Over training is said to have occurred when a model learns the training data too precisely and loses the ability to generalize solutions for the problem. To prevent over training, the entire data was split into three data sets: the training, the validation, and the test data sets. During the training process, the network is tested on the validation sample of cases at each stage of the training and the best set of weights are defined as those that produce the lowest error over the validation data set. After a neural network is trained, we say that the net has learned and that the acquired knowledge has been stored as values in the weights of its connections. In order to test predictive potential of neural network, the test data set (which is not used during the training process) is introduced to the trained network as input. The outp ut is then generated at the output layer and comparing this output with the actual desired response can be used as a tool to evaluate the predictive potential of the trained network. THE DATA Data for this paper is derived from the Computerized Household Activity Scheduling Elicitor (CHASE) survey developed by Doherty and Miller (9). CHASE goes beyond previous methods by provid ing a means to observe the scheduling process as it occurs in reality in a household setting over a multi-day period. In this way it is able to capture both routine and complex scheduling processes as well as observe those scheduling decisions made during the actual execution of the schedule, which have turned out to be substantial. CHASE is a windows-based program that records the sequence of decisions that individuals in a household make to plan and execute their final schedule of activities and travel over a multi-day period. Household members begin the CHASE survey by adding pre-planned activities to their

TRB 2003 Annual Meeting CD-ROM

Paper revised from original submittal.

Doherty and Mohammadian

8

weekly schedule (Monday-Sunday) displayed on a computer similar to a typical dayplanner/calendar, leaving any unplanned time blank (see Figure 3). They are then asked to update their schedule on a daily basis by continually adding, modifying and deleting activities as they are planned and/or executed. Pop-up dialog boxes are used for data entry as shown in Figure 4. The program is designed to record when each of these decisions were made (ranging from pre-planned to impulsive) along with supplemental information prompts on the reasons for certain choices. An initial interview is used to establish basic household characteristics, auto ownership, and a list of potential activity types and their attributes. . The CHASE data used for this paper was collected in the Summer/Fall of 1999 in Quebec City (Canada). Quebec City is the capital city of the province of Quebec and has a population of approximately 170,000. Quebec city includes 15 districts and covers an area of 94 square kilometres. A sample of 36 households were recruited from two residential areas within Quebec City. The sample households represented a mix of single people, couples, couples with pre-teen children, and single parents. In total, the sample included 56 adults and 20 children. A comparison of the sample to the population revealed a slight under-sampling of couples and over-sampling of single people, but was otherwise fairly representative of the population in the sampling areas. Initially, the Quebec City CHASE dataset contained 7405 observed activities and over 140 attributes. After carefully review, a range of activities were removed from the dataset, including: activities planned by household members but not actually performed activities with unknown value of the target variable; and activities missing more than half of all explanatory variables. This left a total of 5583 observed activities for analysis. The target (dependent) variable of concern in this paper in is “planning time horizon”, classified as follows: 1. Preplanned prior to the week, including routinely planned activities 2. Planned on the same day or during the same week - e.g. plan something the morning of, or one or more days in advance, but not prior to the week in question 3. Impulsive planned just before execution ARTIFICIAL NEURAL NETWORK MODEL Neural network models were introduced to transportation engineering in several influential publications, including Faghri and Hua (25), and Yang et al. (26). Since that time, these models have been the topic of substantial theoretical and computational interest and the source of considerable controversy. It has been claimed that any logical function could be duplicated by a suitable network of neurons with thresholds. From this point of view, many transportation or traffic problems can be studied in this modelling framework. Neural networks learn from experience. The models discover a set of connection strengths (weights) that capture the internal structure of a domain. Once a neural network has discovered a set of weights that results in appropriate behavior, the model can repeat a pattern of activity at some later time or produce a pattern of activity when given some portion of it as a cue. Thus,

TRB 2003 Annual Meeting CD-ROM

Paper revised from original submittal.

Doherty and Mohammadian

9

these systems program themselves to perform extraordinarily complex tasks (27). These features of the neural network architecture give rise to properties that make neural networks an attractive alternative to traditional modelling frameworks. In this study, after an extensive search through different algorithms, a well-developed form of ANN, the Multi-Layer (forward-feeding, back-propagating) Perceptron (MLP) was used. Although the main advantage of MLPs is that they are easy to use, and that they can approximate any input/output map, the key disadvantages are that they train slowly, and require lots of training data. Additionally, the required training samples in this type of networks are typically several times more than network weights. The dataset consists of more than 140 attributes for 5583 observed activities. The sample of activities was partitioned to provide a randomly chosen test set of 1675 (30%). The remaining records were further partitioned into a 3350 (60%) record training set, and a 558 (10%) record validation set to be used to test for over fitting in the training process. Extensive testing efforts were undertaken to obtain the best networks in terms of prediction potential. Using genetic algorithm, twenty variables found to be significant in the model with three outputs of pre-planned, planned, and impulsive. Table 1 presents the list of these variables that used in the process of ANN model tuning. There are a number of algorithms that can be used as the underlying method for network design. These include: Multilayer perceptron (MLP), Generalized Regression (GRNN) networks, Principal component analysis (PCA) networks, etc. Table 2 presents a summary of different algorithms tested in this study. As shown in Table 2, the MLP with 62.63% accuracy on test-set, generated the best model fit. This is shown in Table 2 by shaded row. On the other hand, one can notice that the best maximum accuracy attained on the test set for the second alternative (planned) was only 37.81%. This confirmed the initial expectation that this particular category (planned on the same day or during the same week) would be more variable than activities planned at more of the extremes (i.e. preplanned much further in advance, or impulsive). In order to overcome this problem, we decided to combine two alternatives of pre-planned and planned and pursue modeling efforts using only two output alternatives of Pre-Planned and Impulsive. The new pre-planned variable is essentially a combination of both preplanned and planned variables. As a result, total number of independent variables found to be significant in the model (obtained through genetic algorithm) was reduced to 13 variables. Table 3 summarizes the list of these 13 variables. An ANN was developed considering these independent variables and two alternatives of Pre-Planned and Impulsive. Several parameters can be changed during the development of an artificial neural network to improve the performance of the network. These included the number of hidden layers and the number of processing elements in the hidden layer(s), the form of the transfer function (logistic, tanh, or step), and learning rule (momentum vs. step). A summary of the scenarios tested and the results obtained are presented in Table 4. These results were all obtained using the same random training and testing sets. While one parameter was being varied, the others were held at the default values of number of hidden layer, tanh transfer function, and momentum learning

TRB 2003 Annual Meeting CD-ROM

Paper revised from original submittal.

Doherty and Mohammadian

10

method. Information provided in Table 4 represent the results obtained from network parameter tuning using the “test datasets”. Different number of training cycles (epoch) were tested to find the optimum number of epochs. It was felt that 1000 epochs (as suggested by the software) has the best performance since the mean square error (MSE) tends to stabilize after 1000 epochs for almost all scenarios tested. Since genetic trains are very slow, all trainings in this level for network parameter tuning were nongenetic. All trainings were conducted until the error rates for training and validation sets stabilized at approximately the same value. Each 1000 epochs training of the network took slightly more than six minutes on a Pentium Celeron 500 (genetic training of the same network took more than five hours). As expected, the artificial neural network showed some variance in results across the various tuning options. This variation is likely the result of the structure of the problem and the data used. The best scenario that was found to classify the cases more accurately was chosen based on the overall percent predicted correct. This scenario is shown in Table 4 by shaded row (scenario 1). This scenario (one hidden layer, with 22 PEs and tanh transfer function with momentum learning rule) later was used to train the network genetically. A genetic algorithm is used to optimize one or more parameters within the neural network. The most common parameters to optimize are the input columns, the number of hidden PEs, number of memory taps, and the learning rates. Genetic Algorithms are general-purpose search algorithms based upon the principles of evolution observed in nature. Genetic algorithms combine selection, crossover, and mutation operators with the goal of finding the best solution to a problem. They search for this optimal solution until a specified termination criterion is met (28). The solution to a problem is called a chromosome. A chromosome is made up of a collection of genes, which are simply the neural network parameters to be optimized. A genetic algorithm creates an initial population (a collection of chromosomes) and then evaluates this population by training a neural network for each chromosome. It then evolves the population through multiple generations in the search for the best network parameters. In this study 10 generations have been employed for genetic trains while each generation had a population with size of 25 and each population consists of 1000 epochs. The last row in Table 4 (shaded row) summarizes the results of genetic training for the model. The percent predicted correct is 76.32%. While testing the model on the test set, the model predicted 87.15 % of pre-planned activities and 51.66% of impulsive correctly. DISCUSSION AND CONCLUSIONS This paper set out to review the application of an emerging set of machine-learning techniques to an emerging problem – classifying activities according to their planning time horizon (i.e. whether they are preplanned or not). A unique dataset was used that included a wide range of activities and activity attributes, as well as information on when they were planned. Applying machine-learning techniques to this dataset was particular amendable given the wide variety of explanatory variables.

TRB 2003 Annual Meeting CD-ROM

Paper revised from original submittal.

Doherty and Mohammadian

11

The artificial neural network model developed exhibited a relatively high degree of prediction on test data, especially for the extreme categories of planning time horizon (preplanned and impulsive). The result that impulsive decisions were predicted with less accuracy does not come as a surprise – they are more difficult since they tend to depend on several issues in individual’s daily life, all of which were not captured in the survey data (nor perhaps, would people be completely aware of what caused their impulsive decision). While ANN is one of the most commonly used machine learning algorithms, further improvements may be possible using different algorithms (e.g. decision trees, case-based learning). One of disadvantages of ANN techniques is that it does not readily allow evaluation of the role of each explanatory variable in the models to the same extent that traditional statistical-based modes allow. The weights in the first hidden layer (where inputs are explanatory variables) can provide some information regarding the importance (weight) of each variable; however, this is not a practical solution as most ANN software does not provide those weights in a useful format. Additionally, even if we can find the weights of links in the first hidden layer, the effects of weights applied in other hidden layers or output layer might totally change the importance of contributing variables. Thus, interpretation in this paper is limited to examining the overall percentage predicted correct, and generally assessing the potential of such models for the problem at hand. Assessing the role of each variable in the model, and addressing theoretical issues such as the relative contribution of spatial versus temporal fixity in the preplanning of activities is left for further research with a more traditional statistical-based model. Such future efforts may also contribute to the more general debate over the contribution of machine learning models versus traditional discrete choice modelling techniques. It is the authors hope that that the current model serves as a good start to the development of alternative models, and that as more scheduling data becomes available, further model refinements and applications will make for more rigorous debate. Overall, the relative success of the model presented in this paper is promising for emerging activity scheduling process micro-simulation models that are attempting to replicate the sequence with which decisions are made during the scheduling process in order to arrive at observed activity-travel patterns. Part of the motive for these models is the ability to predict how such decisions may be re-sequenced as a result of emerging policies and trends. As a first step, most of these models select an initial set of activities for sequencing, around which subsequent sets of activities are added to complete the scheduling process, mimicking what occurs in reality. This is an important step in any simulation, since initial preplanned decisions constrain later decisions in key ways (e.g. time limits, available modes), which actually makes predicting later decisions more realistic. Instead of assuming that this initial set consists of typical “mandatory” activities (such as work, school), the ANN model developed in this paper could be used in the simulation to select a variety of activities for initial preplanning based on more than just the activity type, and instead allowing variation in preplanned and impulsive activities based on individual, household and activity-based characteristics. This is accomplished by saving the trained network as a dynamic link library (.dll) file which can be simply implemented in the micro -simulation software programs to generate predictions for any given data set. Such a model may be more amendable to different people and places, and would likely be considerably more amendable to handling the

TRB 2003 Annual Meeting CD-ROM

Paper revised from original submittal.

Doherty and Mohammadian

12

types of changes in activities invoked by emerging Transportation Demand Management polic ies. For example, the impacts of telecommunications on “work”(a typically preplanned activity) may be to make it more spatially and temporally flexible, and thereby planed much later in the scheduling process - even planned impulsively at repeated periods throughout the day around other non-traditional preplanned activities. Existing models that make static assumptions about planning order by activity type would have limited means of controlling for such effects.

ACKNOWLEDGEMENTS The authors would like to express thanks to the Social Sciences and Humanities Research Council of Canada, Natural Science and Engineering Research Council of Canada, and Natural Resources Canada for their financial support. Thanks also go to Julie Rozon for her assistance with data collection.

REFERENCES 1. Mohammadian, A. and E. J. Miller. A Comparison of the Performance of Nested Logit Models and Artificial Neural Networks for Predicting Household Automobile Choices . Forthcoming in Transportation Research Record, 2002. 2. Pas, E. I. State of the Art and Research Opportunities in Travel Demand: Another Perspective. Transportation Research, 1985, 19A(5/6): 460-464. 3. Jones, P., F. Koppelman, and J. P. Orfeuil. Activity Analysis: State-of-the-Art and Future Directions, in Developments in Dynamic and Activity-Based Approaches to Travel Analysis, P. Jones, Editor. 1990, Avebury: Aldershot. pp. 34-55. 4. Axhausen, K. W. and T. Gärling. Activity-Based Approaches to Travel Analysis: Conceptual Frameworks, Models, and Research Problems. Transport Reviews , 1992, 12(4): 323-341. 5. Axhausen, K. W. Can We Ever Obtain the Data We Would Like to Have?, in Theoretical Foundations of Travel Choice Modeling , K. Westin, Editor. 1998, Elsevier Science Ltd.: Oxford. pp. 305 -323. 6. Jones, P. M., M. C. Dix, M. I. Clarke, and I. G. Heggie. Understanding Travel Behaviour. 1983, Aldershot: Gower. 281. 7. Recker, W. W., M. G. McNally, and G. S. Root. A Model of Complex Travel Behavior: Part Ii - an Operational Model. Transportation Research, 1986, 20A(4): 319-330. 8. Kawakami, S. and T. Isobe. Development of a One-Day Travel-Activity Scheduling Model for Workers, in Developments in Dynamic and Activity-Based Approaches to Travel Analysis, P. Jones, Editor. 1990, Avebury: Aldershot. pp. 184-205. 9. Doherty, S. T. and E. J. Miller. A Computerized Household Activity Scheduling Survey. Transportation, 2000, 27(1): 75-97. 10. Doherty, S. T., E. J. Miller, K. W. Axhausen, and T. Gärling. A Conceptual Model of the Weekly Household Activity-Travel Scheduling Process, in Travel Behaviour: Patterns, Implications and Modelling, P. Bovy, Editor. 2001, Elgar Publishing Limited: Cheltenham U.K. pp. 148-165. 11. Kitamura, R. Sequential, History-Dependent Approach to Trip-Chaining Behavior. Transportation Research Record, 1983, 944: 13-22. 12. van der Hoorn, T. Development of an Activity Model Using a One-Week Activity-Diary Data Base, in Recent Advances in Travel Demand Analysis, P. Jones, Editor. 1983, Gower: Aldershot. pp. 335-349. 13. Kitamura, R., C. Chen, and R. Pendyala. Generation of Synthetic Daily Activity-Travel Patterns. Transportation Research Record, 1997, 1607: 154-162. 14. Gärling, T., M. P. Kwan, and R. G. Golledge. Computational-Process Modelling of Household Activity Scheduling. Transportation Research , 1994, 28B(5): 355-364. 15. Ettema, D., A. Borgers, and H. Timmermans. Simulation Model of Activity Scheduling Behavior. Transportation Research Record, 1993, 1413: 1-11.

TRB 2003 Annual Meeting CD-ROM

Paper revised from original submittal.

Doherty and Mohammadian

13

16. Gärling, T., T. Kalén, J. Romanus, and M. Selart. Computer Simulation of Household Activity Scheduling. Environment and Planning A, 1998, 30: 665-679. 17. Kwan, M. P. and R. Golledge. Integration of Gis with Activity-Based Model in Atis. in Transportation Research Board 74th Annual Meeting. 1995, Washington, DC: unpublished. 18. Arentze, T. A. and H. J. P. Timmermans. Albatross: A Learning Based Transportation Oriented Simulation System. 2000, Eindhoven, The Netherlands: The European Institute of Retailing and Services Studies. 19. Miller, E. J. and M. J. Roorda. A Prototype Model of Household Activity/Travel Scheduling. in Paper presented at the 82nd Annual Meeting of the Transportation Research Board, Washington, D.C., January 12 16, 2003. 20. Hensher, D. A. and T. T. Tu. A Comparison of the Predictive Potential of Artificial Neural Networks and Nested Logit Models for Commuter Mode Choice. Transportation Research Part E, 2000, 36: 152-172. 21. Neelakanta, P. S., ed. Information-Theoretic Aspects of Neural Networks. 1999, CRC Press: Boca Raton, Florida. 22. Principe, J. C., N. R. Euliano, and W. C. Lefebvre. Neural and Adaptive Systems: Fundamentals through Simulations. 2000, John Wiley & Sons. 23. Picton, P. Neural Networks. Second ed. Grassroots Series. 2000: Palgrave, U.K. 24. Goldberg, D. E. Genetic Algorithms in Search, Optimization, Simulation and Modelling. 1989: University of Alabama. 25. Faghri, A. and J. Hua. Evaluation of Artificial Neural Network Applications in Transportation Engineering. Transportation Research Record, 1991, 1358: 71-80. 26. Yang, H., R. Kitamura, P. P. Jovanis, K. M. Vaugh, and M. A. Abdel-Aty. Exploration of Route Choice Behaviour with Advanced Traveler Information Using Neural Network Concepts. Transportation, 1993, 20(2): 199-223. 27. Parks, R. W., D. S. Levine, and D. L. Long, eds. Fundamentals of Neural Network Modeling. 1998, MIT Press: USA. 28. Lefebvre, C. Neuro Solutions, Version 4.10. 2001.

TRB 2003 Annual Meeting CD-ROM

Paper revised from original submittal.

Doherty and Mohammadian

14

List of Tables and Figures Table 1 Table 2 Table 3 Table 4

Variables used in the ANN model with three alternatives ..............................................15 Results of model testing using different algorithms ........................................................16 Variables used in the ANN model with two alternatives ................................................17 Network parameters tuning result for different scenarios ...............................................18

Figure 1 Typical configuration for a multilayer perceptron and structure of a processing element (PE). ........................................................................................................................................19 Figure 2 Three common transfer functions in neurocomputing. ..................................................20 Figure 3 CHASE main screen, showing example schedule as it might appear on a Friday.........21 Figure 4 CHASE add/modify dialog box .....................................................................................22

TRB 2003 Annual Meeting CD-ROM

Paper revised from original submittal.

Doherty and Mohammadian

15

Table 1 Variables used in the ANN model with three alternatives Explanatory Variable

Acronym

Activity Generic Type (60 types including: Night sleep, Wash, dress, Non-home meals, Eating home-prepared meal, Work, School, etc.) Household years in residence in Quebec region. Number of people in household. Number of children aged 1-12 in household.

HH_RESC HH_SIZE HH_KIDS

Number of teens aged 13-19 in household.

HH_TEENS

Household sample group type (9 groups including: Single and 1 car, Single with kids, Single with teens, Couple and 1 car, Couple and 2 cars, etc.) Number of cars in household. Internet access in household.

A_GEN

HH_STYPE HH_CARS HH_WEB

Household income, average for adults only.

HH_INCX

Individual’s age. Stated frequency of activity per week. Stated normal duration for activity (mins. Activity conducted alone. Activity participated in the local area.

I_AGE A_FREQ A_DURNOR O_INVNON A_PARLOC

Activity Participated in outside local area.

A_PARARE

Activity can be conducted at home Total # locations available for activity Most frequent mode used to get to activity Temporal flexibility/fixity - 0 very flex, 1.0 very fixed. Stated # of days in a week activity could normally occur on.

A_LOCHOM A_LOCTOT A_LOCFR1 A_TIMEFX

TRB 2003 Annual Meeting CD-ROM

A_DAYTOT

Paper revised from original submittal.

Doherty and Mohammadian

16

Table 2 Results of model testing using different algorithms Scenario

Algorithm

Overall percent Percent correct predicted correct Pre-Planned Planned Impulsive

1

Multilayer Perceptron

81.87

37.81

58.04

62.63

2 3 4

Generalized Feed-forward Modular neural network Jordan/Elman Network

82.16 83.17 80.14

26.20 19.82 28.47

58.41 56.19 57.11

59.82 57.85 59.16

5 6

GRNN SOFMs

90.36 86.91

12.07 16.40

34.75 24.21

51.88 48.18

7 8

PCA TRL

79.42 71.80

7.97 20.27

48.06 40.11

50.57 48.06

TRB 2003 Annual Meeting CD-ROM

Paper revised from original submittal.

Doherty and Mohammadian

17

Table 3 Variables used in the ANN model with two alternatives Explanatory Variable Activity Generic Type (60 types including: Night sleep, Wash, dress, Non-home meals, Eating home-prepared meal, Work, School, etc.)

Acronym A_GEN

Number of teens aged 13-19 in household.

HH_TEENS

Household sample group type (9 groups including: Single and 1 car, Single with kids, Single with teens, Couple and 1 car, Couple and 2 cars, etc.)

HH_STYPE

Household income, average for adults only.

HH_INCX

Individual’s age. Stated frequency of activity per week. Stated normal duration for activity (mins). Activity conducted alone. Activity participated in the local area.

I_AGE A_FREQ A_DURNOR O_INVNON A_PARLOC

Activity Participated in outside local area.

A_PARARE

Total # locations available for activity Most frequent mode used to get to activity Temporal flexibility/fixity - 0 very flex, 1.0 very fixed.

A_LOCTOT A_LOCFR1 A_TIMEFX

TRB 2003 Annual Meeting CD-ROM

Paper revised from original submittal.

Doherty and Mohammadian

18

Table 4 Network parameters tuning result for different scenarios

Scen.

# of Hid. # of PEs # of PEs Layers in HL1 in HL2

Transfer Function

1 2

1 1

22 16

0 0

3 4 5

1 1 1

28 22 22

0 0 0

6 7 8

1 2 2

22 22 22

0 11 11

Linear Tanh Sigmoid

9 10

2 1

22 22

11 0

Linear Tanh

11

1

22

0

Tanh

12

1

22

0

Genetic

1

22

0

TRB 2003 Annual Meeting CD-ROM

Tanh Tanh

% correct predicted Overall % Learning Rule Pre-Planned Impulsive correct Momentum 85.52 51.46 75.04 Momentum 84.66 51.46 74.45

Tanh Momentum Sigmoid Momentum Linear Tanh Momentum

87.67 94.66 86.12

41.95 15.53 48.74

73.61 70.33 74.63

Momentum Momentum Momentum

100.0 86.38 100.0

0 49.13 0

69.25 74.93 69.25

100.0 89.83

0 39.03

69.25 74.21

86.47

43.11

73.13

Tanh

Momentum Step Conjugate Gradient Quick Prop

85.87

49.51

74.69

Tanh

Momentum

87.15

51.66

76.32

Paper revised from original submittal.

Doherty and Mohammadian

Input layer

19

Hidden layer

Output layer

PE Z1 Z2 Z3

W1

∑ f(x)

W2

Y

W3

Figure 1 Typical configuration for a multilayer perceptron and structure of a processing element (PE).

TRB 2003 Annual Meeting CD-ROM

Paper revised from original submittal.

20

Logistic Function

Tanh Function

2

2

2

1

1

1

0 -1

f(x)

Step Function

f(x)

f(x)

Doherty and Mohammadian

0

0

-1

-2

-1

-2

x

-2

x

x

Figure 2 Three common transfer functions in neurocomputing.

TRB 2003 Annual Meeting CD-ROM

Paper revised from original submittal.

Doherty and Mohammadian

21

Figure 3 CHASE main screen, showing example schedule as it might appear on a Friday

TRB 2003 Annual Meeting CD-ROM

Paper revised from original submittal.

Doherty and Mohammadian

22

Figure 4 CHASE add/modify dialog box

TRB 2003 Annual Meeting CD-ROM

Paper revised from original submittal.