Collaborative Learning Automata-Based Routing for Rescue ...

IEEE SYSTEMS JOURNAL, VOL. 9, NO. 3, SEPTEMBER 2015

1081

Collaborative Learning Automata-Based Routing for Rescue Operations in Dense Urban Regions Using Vehicular Sensor Networks Neeraj Kumar, Sudip Misra, Senior Member, IEEE, and Mohammad S. Obaidat, Fellow, IEEE

Abstract—In vehicular sensor networks (VSNs), an increase in the density of the vehicles on road and route jamming in the network causes delay in receiving the emergency alerts, which results in overall system performance degradation. In order to address this issue in VSNs deployed in dense urban regions, in this paper, we propose collaborative learning automata-based routing algorithm for sending information to the intended destination with minimum delay and maximum throughput. The learning automata (LA) stationed at the nearest access points (APs) in the network learn from their past experience and make routing decisions quickly. The proposed strategy consists of dividing the whole region into different clusters, based on which an optimized path is selected using collaborative LA having input parameters as vehicle density, distance from the nearest service unit, and delay. A theoretical expression for density estimation is derived, which is used for the selection of the “best” path by LA. The performance of the proposed scheme is evaluated with respect to metrics such as packet delivery delay (network delay), packet delivery ratio with varying node (vehicle) speed, transmission range, density of vehicle, and number of road side units/APs). The results obtained show that the proposed scheme performs better than the benchmark chosen in this study, as there is a 30% reduction in network delay and a 20% increase in packet delivery ratio. Index Terms—Congestion control, learning automata (LA), performance evaluation, routing, vehicular sensor networks (VSNs).

I. I NTRODUCTION

A

N increasing number of current generation vehicles are equipped with advanced wireless technologies to access the network resources on-the-fly and improve the safety of the persons who are riding such vehicles [1]–[4]. These advanced technologies provide vehicle-to-vehicle (V2V) and vehicle-toinfrastructure (V2I) communication infrastructure for a secure and comfortable journey. The passengers in the vehicles have information about the external environment with respect to parameters such as the density of traffic, the distance of the

Manuscript received February 3, 2013; revised April 27, 2014; accepted May 29, 2014. Date of publication August 15, 2014; date of current version June 18, 2015. N. Kumar is with the Department of Computer Science and Engineering, Thapar University, Patiala 147 004, India (e-mail: [email protected]). S. Misra is with the School of Information Technology, Indian Institute of Technology, Kharagpur 721 302, India (e-mail: [email protected]). M. S. Obaidat is with the Department of Computer Science and Software Engineering, Monmouth University, West Long Branch, NJ 07764-1898 USA (e-mail: [email protected]). Digital Object Identifier 10.1109/JSYST.2014.2335451

lights from the current position, the map of the sites to be visited, the quantity of pollution outside the car, and other environment parameters [5], [6]. Vehicles share the information with one another, so that rescue operations can be initiated at once in a particular community, if required. However, due to continuously changing topology with the speed of the vehicles, designing a collaborative routing strategy for vehicular communication is a challenging task [6]. Recently, proposed systems such as FleetNet [7] have investigate the data dissemination and mobility support for efficient communication with discovery of various types of services. FleetNet uses an IPv6-based addressing scheme to connect all the vehicles to the Internet using gateways alongside the road. Some research works [8], [9] in this direction have used the density information on roads to select the optimized routes, but due to the high mobility in the network, sometimes decisions based on statistical data can cause routes to be incorrectly computed. More recently, Yang et al. [4] proposed a scheme, in which the selection of a particular route is decided based upon the transmission quality and the density of the network. The selection of the particular route is done in an optimized manner by selecting the density and duration of the traffic lights. Most of the earlier solutions (e.g., [10] and [11]) may work well for low density areas, but with an increase in the density of the region under investigation, particularly in dense urban regions, it would be a challenging problem to route the packets due to the congestion in the network [12]–[14]. Hence, there is a requirement of an optimized solution, which is adaptive with respect to the topological changes due to the high velocity of the vehicles and generated alerts at constant intervals. Keeping in view the aforementioned challenges and drawbacks in the existing works, we propose a collaborative LAbased routing strategy that can help the community of people to use the rescue operations. In the proposed approach, an automaton learns from its environment and learns the parameters such as vehicle density and distance from road side units (RSUs). Treating these parameters as input variables, the automaton produces an output. The values of these parameters are passed on to the neighboring vehicles in a collaborative manner. The selection of a route depends upon the output produced by the LA by taking into the consideration the vehicle density, and distance from the destination in that region. The rest of this paper is organized as follows. Section II discusses the most relevant related work in this area. Section III describes the background and preliminaries about the theory

1932-8184 © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

1082


of learning automata (LA) in order for the readers to be able to understand the proposed solution. Section IV provides the detailed description of the system model and the problem statement. Section V describes the proposed approach and the algorithm. The simulation results with discussion are presented in Section VI. Finally, Section VII concludes this paper with directions for future work. II. R ELATED WORKS In order to appreciate the contributions of this work, let us review some of the relevant pieces of literature. LeBrun et al. [15] proposed a motion vector routing algorithm (MOVE) to preserve the connectivity of the vehicle with an RSU. In this paper, the authors proposed the use of the neighboring vehicle velocity to predict the closeness of the traveling vehicle with the others. Zhao and Cao [16] proposed a vehicle-assisted data delivery (VADD) protocol for vehicular ad hoc networks (VANETs). The authors proposed a new metric called the expected delivery delay metric to guide during the routing process. However, the proposed scheme has limited performance in situations where an adaptive decision needs to be taken based upon the input parameters such as the density of nodes and the distance. Fallah et al. [17] proposed a scheme for a vehicle safety system, in which they proposed an adjustment of rate and range parameters of the vehicles. The proposed scheme was evaluated in different network environments with respect to various evaluation parameters and its performance was found to be satisfactory in comparison to other existing schemes. Lee et al. [18] proposed a middleware approach for urban monitoring using sensor nodes. The authors proposed an analytical model by taking various scenarios of mobility and stability of the sensor nodes. The performance of the proposed scheme was better than the other existing schemes. Rawat et al. [19] proposed an approach, in which power and size of the contention window were changed for service differentiation. In their approach, the authors compute the local density, and then, the transmission range is made adaptive using this parameter. Moez et al. [20] proposed an efficient routing protocol for dense urban regions. In their work, the distance from the final destination is first computed, and that value is used to find an optimized path in dense urban region. Jeong et al. [21] proposed a scheme, in which the vehicles on the road calculate their data delay to the access points (APs) in their regions, and then, this information is shared with the other vehicles. Tatchikou et al. [22] proposed a collision avoidance scheme using VANETs. The authors proposed a collision avoidance scheme using an efficient packet forwarding mechanism. Buchenscheit et al. [23] proposed a warning system for rescue operations using VANETs. In the proposed scheme, the vehicles share the information about the best suitable path. Fiore and Barcelo-Ordinas [24] proposed a scheme for optimized AP deployment and for download of data from these APs. Zhou et al. [25] proposed a routing scheme for distributed media services for VANETs in a peer-to-peer manner. The authors proposed an optimization model for data transfer and cache update for user satisfaction. Burgess et al. [26] proposed a routing scheme for vehicle-based disruption-tolerant networks.

Fig. 1.

LA. Automaton interaction with an environment.

The proposed scheme was based on prioritization and scheduling of routing packets. Zhou et al. [27] proposed a cross-layer routing design for VANETs. The proposed scheme establishes end-to-end connectivity and semantics for various applications in VANETs. Leontiadis and Mascolo [28] proposed routing scheme by taking into consideration the mobility and location of the vehicles in VANETs. Zhang and Wolff [29] proposed a routing scheme for VANETs for use in those regions where fewer infrastructures exist for the vehicles to communicate with one another. It has been found in literature that computational intelligence techniques can be useful to solve various complex engineering problems from diverse backgrounds [30]–[39]. All these problems are solved by using the concepts of learning and interactions of objects with an environment that provides constant feedback to these objects such that these can take adaptive decisions. Learning can guide and improve the routing criteria in many adaptive systems [30]–[39]. III. BACKGROUND AND P RELIMINARIES Fig. 1 describes the working of LA. An automaton takes the input parameters as previously defined and acts according to these parameters to produce an output. It is an adaptive learning technique with decision-making machine having the capability of improvement by learning from its environment so that it can choose the optimal action from a finite set of allowed actions through repeated interactions [30]–[39]. The objective of a learning automaton is to find the optimal solution with minimized penalty received from the environment [30]–[39]. Mathematically, LA is defined as Q, K, P, δ, ω, where Q = {q1 , q2 , . . . , qn } are the finite set of states of LA; K = {k1 , k2 , . . . , kn } are the finite set of actions performed by the LA; P = {p1 , p2 , . . . , pn } are the finite set of response received from the environment; and δ : Q × P → Q maps the current state and input from the environment to the next state of the automaton; and ω is a function that maps the current state and response from the environment to the state of the automaton [30]–[39]. The environment in which the automaton operates can be defined as a triplet X, Y, ρ, where X = (X1 , X2 , . . . , Xn ) are finite number of inputs; Y = (Y1 , Y2 , . . . , Yn ) are values of reinforcement signal; and ρ = (ρ1 , ρ2 , . . . , ρn ) are penalty probability associated with each Xi , 1 ≤ i ≤ n. The automaton performs a finite number of actions, and based upon their actions, the response of the environment can be either a reward or

KUMAR et al.: CLARA FOR RESCUE OPERATIONS IN DENSE URBAN REGIONS USING VSNs

Fig. 2.

1083

System model in the proposed scheme.

a penalty. According to the response received from the environment, automaton decides its action by taking the reinforcement signal to which stage it has to move. Corresponding to each input parameter and action, automaton updates its action probability vector by using the learning algorithm. In the proposed scheme, we have considered linear reward-inaction scheme, in which if automaton receives reward from the environment then action probability is updated, or else probability remains same [30]–[39]. The formula for probability update is as follows: pj (n + 1) = (1 − a)pj (n), pj (n + 1) = apj (n), pj (n + 1) = pj (n),

j = i,

j = i,

Y =0

Y =0

Y =1

where a is a parameter. IV. S YSTEM M ODEL AND P ROBLEM F ORMULATION Fig. 2 shows the system model in the proposed scheme. A VSN consists of nnumber of nodes with every node having a unique identity. It is assumed that every node is equipped with a Global Positioning System (GPS), digital map, wireless transmitter, video camera, and storing device. As shown in Fig. 2, there are different communities on each side of the road. These communities are connected to the Internet using a gateway via the service providers (SPs) in their vicinity. There are two types of communications considered in the proposed scheme: V2I and V2V. For V2I, one RSU/AP is deployed in different regions of the area to be investigated. The RSU sends/receives the information from gateway to SP to the community of peoples. For the V2V case, each vehicle communicates and

Fig. 3. (a)–(b). Route Selection with (a) collaborative information sharing and (b) density estimation in the proposed scheme.

shares the information with the other vehicle in peer-to-peer manner. A road is divided into clusters, which are aligned in a single row. Fig. 3 shows the relationship between velocity and density of the network. With an increase in the density of the network, the average velocity of the vehicle decreases due to traffic congestions and an increase in collision probability. Fig. 3(a) shows how the interaction between different states where automaton is working is done, while Fig. 3(b) shows how the distance from the RSU influences the route selection process. Table I shows the symbols and their meaning.

1084


TABLE I S YMBOLS AND TABLES U SED

V. P ROPOSED A PPROACH The proposed approach consists of the route establishment and route selection phases. The route establishment phase consists of construction of routes using LA, whereas the route selection phase consists of intelligent route selection, again using an LA-based approach. The detailed discussion about the two phases is as follows: A. Route Establishment

A. Problem Statement Let L = {L1 , L2 , . . . , Ln } be the current location set of the vehicles;θ = {d1 , d2 , . . . , dn }, the distance from the neighboring RSU; D = {k1 , k2 , . . . , kn } the vehicle density set; and R = {R1 , R2 , . . . , Rn } the current routes set. The current location of the vehicle can be found at any time by using GPS. To make a transition from one state to another, we consider a transition function as defined in the following: δij : x → R

(1)

where x = (L, θ, D) are input parameters, 1 ≤ i ≤ n, 1 ≤ j ≤ n The right-hand side of (1) contains the possible routes and δij is a stochastic function, which acts according to the input parameters. It maps the current location of the vehicle, the distance from the RSU and the density of the vehicle to suitable route. The density of the vehicle is measured by the number of vehicles in a particular cluster/region to the total number of vehicles at any instant and is calculated by (2) as defined in the following: n N Vi i=1 (2) D= n ni i=1

where N Vi is the number of vehicles in cluster i, and ni the total number of vehicles on the road. We have set a predefined threshold range of 300 m for computing the number of vehicles in a region. Each cluster has its own density of vehicles, and hence, proper routing mapping is performed using the mapping of input parameters to the current route using the values in transition probability matrix (TPM) defined as in the following. The values in this matrix give the probability of selection of optimized route for the data dissemination. The associated TPM can be defined as follows: ⎤ ⎡ δ12 −−− δ1n δ11 δ22 −−− δ2n ⎥ ⎢ δ21 (3) ⎦ ⎣ −−− −−− −−− −−− δn2 −−− δnn δn1 where δij is defined by (1). Hence, our goal for the given problem is to find the optimized route with minimum weight with parameters as defined in the coming section.

A packet containing the parameters L, θ, D, δt is transmitted to the intermediate node having information about the vehicle current position, distance from RSUs, vehicle density, and total delay incurred in a particular region. This phase is completed by the automaton and corresponding to each action taken by the automaton, it is penalized or rewarded by some constant value based upon which the automaton decides its next action. The path from source to destination is constructed in different stages according to the transition function δij . The values of δij are changed according to the reward, penalty, and action probability n_p vector of an automaton as follows. THRi = (1/n_p) i=1 Wi (t), where THRi is the threshold for path i, which determines the packet delivery capability of that path, n_p is the number of paths from source to destination, and Wi (t) is the cumulative weight of these paths as follows: N (n_p) Wi (t) = (1/N (n_p)) j=1 W (j), where N (n_p) is the number of times weight along that path has been computed, and W (j) is the weight of an individual path. Now, based upon the aforementioned, two types of action are possible for the automaton as reward and penalty. In the reward function, if the current node is the destination, then cumulative weight is updated as aforementioned with ψ ∈ [0, 1] as a reward constant. Otherwise, the transition function is used for update the cumulative weight as aforementioned. ξ is a constant to estimate how good is the path selected. A higher value of ψ indicates faster convergence toward the optimized path and lower value indicates slow convergence toward the optimized path. Similar to the reward function, a penalty is associated, as shown in the following. In the aforementioned, φ ∈ [0, 1] is the penalty parameter, which decreases the cumulative weight for every unsuccessful transmission. The specific values of φ in this interval is taken to minimize the effect of penalty for unsuccessful transmission. As for every unsuccessful transmission, action of automaton is penalized so we have taken values of φ as previously defined to minimize the penalty. Based on the aforementioned, every automaton at respective nodes selects or rejects a packet based upon THRi and updates its action probability vector as previously defined. If the newly arrived packet has higher distance and density than the defined THRi of the packets, which has been received in the past, then the status of the current packet is declared to be rejected; otherwise, if the newly arrived packet has the same distance and density for the packets, which has been received in past by the intermediate node, then it is added in the path for the final destination and can be used in future. Furthermore, if the newly arrived packet has lower distance and density than the defined THRi of the packets, which has been received in the past, then this packet is accepted by the automaton. In such a case, all the


previously found paths are discarded and the newly found path is considered for inclusion in the final path. We have assumed the clustering of the nodes as described in [6], [9]. Algorithm 1 Reward function if (current_node = destination) then Wi (t) = Wi (t) + ψ else δij = ξ × Wi (t) + (1 − ξ)Wi (t) end if

Algorithm 2 Penalty function if (current_node = destination) then Wi (t) = Wi (t) × φ else Wi (t) = Wi (t) + ψ δij = ξ × Wi (t) + (1 − ξ)Wi (t) end if

As shown in Fig. 3(a), the information is shared among the participating vehicles in a collaborative manner. Based upon this information, the routing packet will be transmitted to a suitable node in a collaborative manner. There are actions set at each node, which will be taken by LA from A = {A1 , A2 , . . . , Ak } based upon the input values. The time space is divided into different intervals and accordingly the action taken by LA at a particular instant of time is rewarded or penalized by the environment, in which it is operating. In addition, the value learnt by the LA is affected by the reinforcement signal. Let Ak (n + 1), Ak (n), Al (n + 1), Al (n) be the various action states at different time intervals. If the newly arrived state has higher density than the current state, then LA will not select that state but keep it in the current state; otherwise, it will select the next destination intelligently based upon the minimum density and maximum value of entries from TPM. As we have assumed that the whole road is divided into different clusters, so the probability of the vehicle in a particular cluster at any time t is defined as p(D, t) = p(f, Δt), where Δt ∈ (t, t + dt).

For example, if we have taken 20 individual paths, then all of these paths have same probability of selection 1/10 = 0.1 (for example). Let us have computed weight along the path ten times, then the reward function if the current node is the destination node, can be set as Wi (t) = 1/10 × 20 × 0.1 + 0.5 = 0.7. Similar computation can be done for penalty function also. B. Route Selection Once the route establishment phase, the next step is to select the best neighbors from the routing list using an LA-based approach. Each node learns about its best neighbor, which has the minimum value of the parameters θ, D, so that routing packets can be sent to these neighbors. In Fig. 3, we represent the different states of the moving vehicle and selection of next hop intelligently using the estimation of the vehicle density. As shown in the figure, the selection of the next hop is probabilistic in nature. The whole duration of inspection is subdivided into different time intervals of equal duration and intensity of the vehicle is calculated during each time interval by estimation of vehicle speed with respect to different parameters such as density and distance from the final destination. During the start of the next time interval, vehicle speed may vary according to the density of the traffic. Hence, estimation of the density of a vehicle in a particular time interval is required so that LA can intelligently select the best path. Let v be the speed of the vehicle in a particular time instant t and v ± dv, its speed in the next time instant t + dt. During any time interval, the distance between the adjacent vehicles follows an exponential distribution with an intensity [40], as shown in +∞ f dv ρ=λ v

1085

(4)

−∞

where λ is a rate parameter, and f is a probability density function (PDF) of the speed distribution.

(5)

The PDF of the vehicle speed v with standard deviation σ, can be defined using Gaussian speed distribution [41] as follows:

−v 2 1 2σ 2 . (6) PDF = √ e σ 2π If we represent the position of RSU as (x0 , y0 ) and (xi , yi ) as the position of a particular vehicle, 1 ≤ i ≤ n, then f can be formulated [43] as follows: ⎛ r r ⎞ n ⎝ (xi,n − x0 )2 − (yi,n − y0 )2 ⎠ dx dy. (7) f= i=1

0

0

Inserting the value of (7) in (5), we get the probability of the vehicle in a particular cluster i at any instant of time. This is applicable to all the vehicles traveling on the road. This information is shared by each vehicle with the other in a collaborative manner, as shown in Fig. 5(a). This density information varies with time and is used as an input parameter by LA to adaptively select a path that is less dense. It will then have less chance of collision and, at the same time, higher chances of packet transmission to the desired destination. Lemma 1: If λ is the rate at which a vehicle comes in a cluster, E is the mean time by LA to perform an action in the cluster, and R is the total radius of the cluster, in which LA performs an action, then the change in the density of the vehicles in that cluster will be R −1 Dw Di (1 − λij ), w = 0, 1, 2, . . . , R (8) ΔD = w! i=0 i! where λij , 1 ≤ i, j ≤ n are the arrival rates of vehicles in i × j array of x–y position Proof: Assume that λ is the rate at which a vehicle comes in a cluster with the M/G/R/R queuing model with E is the mean time by LA to perform an action in that cluster, and R is the total radius of the cluster, in which LA perform an

1086


action. Then, the probability pn of availability of the vehicle in a particular cluster is given by [40], [41] R −1 ρn ρi , pn = n! i=0 i!

probability vector of LA is updated according to the as previously defined, then for every > 0, ∃ learning rate a ∈ (0, 1) such that ∀a ∈ (0, 1), we have Prob[ lim (pi (l) = 1)] ≥ 1 − . l→∞

n = 0, 1, 2 . . . , R,

ρ = λE. (9)

Probability pn changes with time as the new vehicle arrives and leaves the cluster in a particular time interval. Hence, accordingly, density D of the vehicles also changes. Thus, we represent the change in the density of the vehicle as ΔD. The value of ΔD is dependent on the vehicle arrival rate and their mean time to stay in a particular cluster R −1 ρn ρi pn (R, ρ) = n! i=0 i!

(10)

R −1 ρn ρi pn (R − 1, ρ) = n! i=0 i!

(11)

.................................. R −1 ρn ρi pn (0, ρ) = . n! i=0 i! Subtracting (10), (11), and so on, in pairs, we get, ΔD, as follows: ΔD = (pn (R, ρ)) − (pn (R − 1, ρ)) −, , , − (pn (R − 1, ρ)) − (pn (0, ρ)).

(12)

Equation (12) gives an estimate of the change in the density of the vehicle in a particular region. This density calculation in (12) is multiplied with the probability of route selection by TPM. Hence, replacing the ρ by D and n by W R −1 Dw Di . (13) ΔD = w! i=0 i! Multiplying (13) by the values of TPM, we get the desired expression. The algorithm 3 consists of estimation of vehicle density, distance from the neighboring RSU and delay incurred using (4)–(7). If the newly calculated density and distance from the RSU are more than the old, then the arrived packet at the intermediate node is discarded; otherwise, the packet is kept as repository for future. Moreover, if this value is less than the old value, then the routing table is updated by adding this new value and all the previous paths are discarded. In the second step, LA perform an action by selecting a neighboring node with minimum value of density and distance from RSU. Based upon the action taken by LA, its action from the action set A = {A1 , A2 , . . . , Ak } can be or penalized (lines 16–21). The maximum value of ratio between the numbers of rewards to penalty is taken (lines 28–30). Theorem 1: Let pi (l) be the probability of selection of values from TPM at any path at any stage l by the LA. If the action

Proof: Proof of theorem is provided in theAppendix. Algorithm 3 Collaborative Learning Automata based Routing Algorithm (CLARA) Inputs: Set of actions as V : {V1 , V2 , V3 } where V1 = drop_result V2 = add_as_alternate_path V3 = add_routing_table Output: Optimized route selection Parameters: L, θ, D, δt, R Q = {φ(reward), Z(penalty)} q : Index of minimal value of (θ, D) Mi : The number of times an action i is rewarded Ki : Total number of times action i has been taken Route Establishment 1: Source broadcast the packet with L, θ, D, δt 2: The automaton at respective node acts according 3: if ((θnew &&Dnew ) > (θold &&Dold )) then 4: Discard (V1 ) the newly arrived packet 5: else 6: if ((θnew &&Dnew ) = (θold &&Dold )) then 7: Add to the routing table as an alternative path(V2 ) 8: end if 9: else 10: if ((θnew &&Dnew ) < (θold &&Dold )) then 11: Add the newly found path (V3 ) in the routing table 12: end if 13: end if 14: Discard all the previously find paths 15: Update the action probability vector 16: Route Selection 17: Find the location of the vehicle node using GPS 18: Create a packet with information L, θ, D, δt. 19: LA performs action with min(θ, D). 20: The feedback variable y is given to LA. 21: if ((|δij − δij | < )&&(y = 1)) 22: Perform the action Ak for reward as 23: Ak (n + 1) = max(Ak (n), φ(δi,j )). 24: Al (n + 1) = max(Al (n), (1 − φ)(δi,j )), ∀ll = k 25: else 26: Ak (n + 1) = (1 − Z)(1 − δij )Ak (n) 27: Al (n + 1) = Z(1 − δij )Al (n), ∀ll = k 28: Update the values of LA in action probability 29: g(n + 1) = Ak (n + 1)/Al (n + 1) 30: Choose the maximal value of g(n + 1) 31: end if 32: Alert generation 33: Perform the action from set V 34: Choose the best hop as the next destination 35: Generate the alerts and send the suitable destination 36: Repeat steps (1–31) until no further rescue requests arrive.


1087

TABLE II S IMULATION PARAMETERS

Fig. 5. Impact on packet delivery ratio.

Fig. 4.

Network delay with variation in density.

VI. P ERFORMANCE E VALUATION This section illustrates the performance evaluation of the proposed scheme in comparison to other state-of-the-art existing schemes in literature. Fig. 6. Network delay with variation in vehicles speed.

A. Simulation Environment The proposed scheme was evaluated on VANET MobiSim [42] and its performance was compared with the existing competing schemes: MOVE [15] and VADD [16]. Each vehicle’s movement pattern was determined by a Hybrid Mobility model [43]. The packet generations from the vehicles were considered to be an exponential distribution with a mean of 10 s; packets were dynamically generated from 500 vehicles in the road network. The simulation parameters that were considered in the study are described in Table II. B. Results and Discussion 1) Impact of Node Density on Network Delay: Fig. 4 shows the impact of the node density on the network delay in the proposed scheme. As shown in the figure, with an increase in the number of nodes in the network, the network delay in the proposed scheme decreases compared with the benchmark schemes. This is due to the fact that the proposed scheme uses an LA-based approach, which intelligently adapts to the situations such as congestion/jamming quickly and takes the decision on its own, which results in a considerable decrease

in the network delay compared with the other approaches in its category. 2) Impact of Node Density on Packet Delivery Ratio in the Proposed Scheme: Fig. 5 shows the impact of the proposed scheme on the packet delivery ratio. The results show that the proposed scheme has a higher packet delivery ratio than the other two schemes with the same density level. As shown in the figure, the “gap” between the plots corresponding to the obtained packet delivery ratio in the proposed scheme and the other two schemes becomes wider. The reason for such behavior with the same level of density of nodes is due to the fact that the proposed scheme intelligently chooses the route (adaptively) from the available ones using an intelligent LA approach that learns with the passage of time. Thus, the decision taken by LA for the selection of best path in the proposed scheme is better than the other existing schemes reported in the literature. As a result of this, the packet delivery ratio in the proposed scheme increases considerably compared with the other approaches. 3) Impact of Node Speed on Network Delay in the Proposed Scheme: Fig. 6 shows the impact of the proposed scheme on network delay with an increase in node speed. As shown in the

1088

Fig. 7. Packet delivery ratio with an increase in vehicles speed.

figure, with an increase in the node speed, the network delay also increases. This is due to the fact that with an increase in the node speed, the distance between vehicles also increases and, consequently, it is difficult to maintain the connection between the nodes, thereby resulting in an increase in the network delay. However, connectivity is efficiently maintained in the proposed scheme with an increase in node speed due to the selection of adaptive paths intelligently using a LA-based approach. As a result of this, the network delay is minimized in the proposed scheme compared with the others, as shown in Fig. 8. 4) Impact of Node Speed on Packet Delivery Ratio: Fig. 7 shows the impact of the proposed scheme on the packet delivery ratio with an increase in the node speed for 500 nodes. The results obtained show that the proposed scheme has the highest packet delivery ratio compared with the other competing schemes with an increase in node speed. Although with an increase in node speed, the packet delivery ratio drops considerably in all three schemes, but this drop is relatively less prominent in our proposed scheme compared with other two schemes. This shows the effectiveness of the proposed scheme over the other two schemes. Again, the reason for a consistent increase in packet delivery ratio value in the proposed scheme is its ability to choose the path using a LA-based approach. The learning automaton intelligently selects the best path from the available ones and, hence, with an increase in node speed, almost the same level of packet delivery ratio is maintained, which drops considerably less than the other two schemes. 5) Impact of Density of the Network on Overhead Generated: Fig. 8 shows the impact of variation of density on overhead generated. As shown in the figure, with an increase in the density of the nodes in the network, the overhead generated in the proposed scheme is less as compared with the other approaches. This is due to the fact that the proposed approach uses LA, which executes the operation of route establishment and data dissemination in cooperation with the other automata. The overhead generated is mainly due to the control messages transferred from one node to the other, but in the proposed scheme, whole operations are controlled by the respective automata. None of the other schemes has used an optimized procedure to calculate the overhead generated. Hence, there


Fig. 8.

Overhead generated with variation in the density.

Fig. 9.

Probability of alerts transfer.

is an increase in the overhead generated in these schemes as compared with the proposed approach. 6) Probability of Alerts Transfer With Node Density: Fig. 9 shows the impact of the node density on the probability of alerts transfer to the destination. As shown in Fig. 9, with an increase in the node density, there is a decrease in the probability of alerts transfer to its final destination. This is mainly due to the reason that at higher density of the vehicles, the chances of occurrence of collision is also get increased, which results a decrease in the probability of alerts transfer. But as the proposed approach used an intelligent approach, in which LA take the decisions based upon their past behavior, so chances of alerts transfer to their final destination are also high. Hence, the proposed scheme performs better than the other algorithms taken in this study. VII. C ONCLUSION In this paper, we have proposed a new collaborative LAbased routing scheme that can help in rescue operations for dense urban regions using VSNs. Each moving vehicle has


an intelligent sensor deployed, which selects the route adaptively and intelligently, based on the density of the vehicle on the road, the distance from RSUs and delay incurred during transmission. Vehicles communicate with one another in a collaborative manner in order to share the information about these variables and intelligently select the best route to reach the final destination. LA selects the best path from the available ones. The performance of the proposed approach was evaluated along with the existing competing approaches with respect to performance metrics of packet delivery ratio and packet delivery delay by varying the node speed, the density of the vehicles, and the transmission ranges. The results obtained show that the proposed scheme shows superior performance as it shows a reduction of 30% in the network delay and an increase of 20% in packet delivery ratio compared with the existing schemes that we considered in this study. In the future, we plan to study the effectiveness of the proposed solution on real test beds. We also plan to focus on the cache consistency and maintenance for adaptive data dissemination in VSNs. Various contextual cache replacement strategies would also be explored. A PPENDIX Theorem 1: Let pi (l) be the probability of selection of values from TPM at any path at any stage l by the LA. If the action probability vector of LA is updated according to the as previously defined, then for every > 0, ∃ learning rate a ∈ (0, 1) such that ∀a ∈ (0, 1), we have Prob[ lim (pi (l) = 1)] ≥ 1 − l→∞

Proof: The proof of the theorem consists of following three steps, which are described as follows: 1) as l → ∞, the penalty probability of LA approaches to a constant value (Lemma 2); 2) as l → ∞, the probability of selection of TPM with respect to request arrival is approached to a maximum value; 3) finally, the convergence of CLARA is proved. Lemma 2: If the action of LA in CLARA is penalized with the probability Zi (l) at any stage l such that [ lim (Zi (l) → l→∞

Zi (l)opt )], then ∀ ∈ (0, 1), we have Prob|Ziopt − Zi (l)| > , where Ziopt is the optimal value. Proof: The idea of convergence of probability of selection of particular action to an optimal value by LA is given in [30]– [39]. We have applied the same concept in different problem, in which the goal is to maximize the value in TPM along the path using LA. As we are using model in the current proposal, so in case of penalty the probability does not change. As l → ∞, Zi (l) and Zi (l)opt approaches to the closer so the probability of penalizing Zi (l)opt is decreased. As l → ∞, Prob|Ziopt − Zi (l)| > . Hence, the proof of Lemma 2. Lemma 3: If Pit is the probability of selection of action ki at time t and Pjt is the probability of selection of action kj at time t by LA, respectively corresponding to the mean expected number of requests arrival, then the maximum TPM along a chosen path by LA can be achieved as: TPMmax ≤ pti ((E(t)(1 − (E(t))n+1 )/1 − (E(t)))) and, where E(t) are the expected number of vehicles arrival.

1089

Proof: The velocity of the vehicles is changed due to vehicles arrival (β) and departure (δ). If f (t) is the PDF, then Laplace transform for vehicle arrival can be given as follows: +∞ +∞ −βt 2 f (t)e dtf (t) = f (t)e−δt dt. f (t) = 1

0

(14)

0

Let Pit is the probability of selection of action ki from the set K at time t by LA corresponding to the requests arrival. Moreover, let η = (η1 , η2 , . . . , ηn ) are the rewards received corresponding to the action taken, which are symmetrical [44]. The expected numbers of vehicle arrival are as follows: ⎛t 2 t4 t⎝ −βt f (t)e dt f (t)e−βt dt E(t) ≤ Pi t1

t3

n

+··· +

⎞ f (t)e−βt dt⎠ .

(15)

n−1

The aforementioned equation in its modified form can be rewritten as ⎛ 2t 2 2t4 2 t⎝ −βt E (t) ≤ Pi f (t)e dt f (t)e−βt dt 2t1

⎞

2t3

2n +··· +

⎟ f (t)e−βt dt⎠

(16)

2(n−1)

...⎛ ........................... nt2 nt4 n t⎝ −βt f (t)e dt f (t)e−βt dt E (t) ≤ Pi nt1

nt3 n∗n

+··· +

⎞ ⎟ f (t)e−βt dt⎠ .

(17)

n(n−1)

Then the mean arrival and departure can be given as follows: TPMmax ≤ Pit E(t) + E 2 (t) + · · · + E n (t) E(t) 1 − (E(t))n+1 . ≤ Pit 1 − (E(t))

(18) (19)

The values of Pit and Pjt are dependent on how many number of times an action taken by LA is rewarded or penalized. Hence, it can be formulated as follows:

NR NR Pit = ψ− φ (20) N (n_p) N (n_p) where NR are the number of time action taken by LA is rewarded and NP is the number of times action is penalized. As n → ∞, Pit → 1 Pjt → 1 TPMmax ≤ ((E(t)/1 − (E(t)))) f (t) in can be computed as follows: f (t) = μx e−μ /x!, e.g., for x = 0, μ = 2, f (t) = 0.135 (Vehicles arriving in one minute (probability 0), with flow rate μ = 3) for x = 1, μ = 2, f (t) = 0.271 (Vehicles arriving in one minute (probability 1), with flow rate μ = 2). We get the values of TPM approaching to 1

1090


(close to 1). Hence, third part of the theorem is proved and we can say that proposed CLARA algorithm is − optimal, i.e., Prob[liml←∞ (pi (l) = 1)] ≥ 1 − R EFERENCES [1] A. Balasubramanian, R. Mahajan, A. Venkataramani, B. N. Levine, and J. Zahorjan, “Interactive WiFi connectivity for moving vehicles,” SIGCOMM Comput. Commun. Rev., vol. 38, no. 4, pp. 427–438, Oct. 2008. [2] J. Eriksson, H. Balakrishnan, and S. Madden, “Cabernet: Vehicular content delivery using WiFi,” in Proc. 14th ACM Int. Conf. MobiCom Netw., San Francisco, CA, USA, Sep. 14–19, 2008, pp. 199–210. [3] A. Dua, N. Kumar, and S. Bawa, “A systematic review on routing protocols for vehicular ad hoc networks,” Veh. Commun., vol. 1, no. 1, pp. 34– 52, Jan. 2014. [4] Q. Yang, A. Lim, S. Li, J. Fang, and P. Agrawal, “ACAR: Adaptive connectivity aware routing for vehicular ad hoc networks in city scenarios,” Mobile Netw. Appl., vol. 15, no. 1, pp. 36–60, Feb. 2010. [5] N. Kumar and J. H. Lee, “Peer-to-peer cooperative caching for data dissemination in urban vehicular communications,” IEEE Syst. J., vol. 8, no. 4, pp. 1136–1144, Dec. 2014. [6] N. Kumar, N. Chilamkurti, and J. H. Park, “ALCA: Agent learning-based clustering algorithm in vehicular ad hoc networks,” Pers. Ubiquitous Comput., vol. 17, no. 8, pp. 1683–1692, Dec. 2013. [7] FleetNet Homepage 2006. [Online]. Available: http://www.et2.tuharburg.de/fleetnet/index.html [8] Q. Yang, A. Lim, and P. Agrawal, “Connectivity aware routing in vehicular networks,” in Proc. IEEE WCNC, Las Vegas, NV, USA, Mar. 31–Apr. 3 2008, pp. 2218–2223. [9] A. Daeinabi, A. Rahbar, and A. Khademzadeh, “VWCA: An efficient clustering algorithm in vehicular ad hoc networks,” J. Netw. Comput. Appl., vol. 34, no. 1, pp. 207–222, Jan. 2011. [10] V. Namboordiri and L. Gao, “Prediction-based routing for vehicular ad hoc networks,” IEEE Trans. Veh. Technol., vol. 56, no. 4, pp. 2332–2345, Jul. 2007. [11] R. Sengupta et al., “Cooperative collision warning systems: Concept definition and experimental implementation,” J. Intell. Transp. Syst., vol. 11, no. 3, pp. 143–155, Jul. 2007. [12] C. L. Huang, Y. P. Fallah, R. Sengupta, and H. Krishnan, “Information dissemination control for cooperative active safety applications in vehicular ad-hoc networks,” in Proc. IEEE GLOBECOM, Honolulu, HI, USA, Nov. 30–Dec. 4 2009, pp. 1–6. [13] C. L. Huang, Y. P. Fallah, R. Sengupta, and H. Krishnan, “Adaptive inter vehicle communication control for cooperative safety systems,” IEEE Netw., vol. 24, no. 1, pp. 6–13, Jan./Feb. 2010. [14] S. Rezaei, R. Sengupta, H. Krishnan, X. Guan, and R. Bhatia, “Tracking the position of neighboring vehicles using wireless communications,” Transp. Res. C, Emerging Technol., vol. 18, no. 3, pp. 335–350, Jun. 2010. [15] J. LeBrun, C. N. Chuah, D. Ghosal, and M. Zhang, “Knowledge-based opportunistic forwarding in vehicular wireless ad hoc networks,” in Proc. IEEE VTC, Stockholm, Sweden, May 30–Jun. 1 2005, pp. 2289–2293. [16] J. Zhao and G. Cao, “VADD: Vehicle-assisted data delivery in vehicular ad hoc networks,” IEEE Trans. Veh. Technol., vol. 57, no. 3, pp. 1910– 1922, May 2008. [17] Y. Fallah, P. Huang, C. Ling, R. Sengupta, and H. Krishnan, “Analysis of information dissemination in vehicular ad-hoc networks with application to cooperative vehicle safety systems,” IEEE Trans. Veh. Technol., vol. 60, no. 1, pp. 233–247, Jan. 2011. [18] U. Lee, E. Magistretti, M. Gerla, P. Bellavista, and A. Corradi, “Dissemination and harvesting of urban data using vehicular sensing platforms,” IEEE Trans. Veh. Technol., vol. 58, no. 2, pp. 882–901, Feb. 2009. [19] D. B. Rawat, D. Popescu, C. Yan, and O. Gongjun, “Enhancing VANET performance by joint adaptation of transmission power and contention window size,” IEEE Trans. Parallel Distrib. Syst., vol. 22, no. 9, pp. 1528– 1535, Sep. 2011. [20] J. Moez, S. Mohammed, R. Tinku, and D. Y. Ghamri, “Towards efficient geographic routing in urban vehicular networks,” IEEE Trans. Veh. Technol., vol. 58, no. 9, pp. 5048–5059, Nov. 2009. [21] J. Jeong, S. G. Guo, J. Yu, H. D. Tian, and H. C. David., “Trajectory-based data forwarding for light-traffic vehicular ad hoc networks,” IEEE Trans. Parallel Distrib. Syst., vol. 22, no. 5, pp. 743–757, May 2011. [22] R. Tatchikou, S. Biswas, and F. Dion, “Cooperative vehicle collision avoidance using inter-vehicle packet forwarding,” in Proc. IEEE GLOBECOM, St. Louis, MO, USA, Nov. 28–Dec. 2 2005, pp. 2762–2766.

[23] A. Buchenscheit, F. Schaub, F. Kargl, and M. Weber, “A VANET-based emergency vehicle warning system,” in Proc. 1st IEEE IEEE VNC, Tokyo, Japan, Oct. 28–30, 2009, pp. 1–8. [24] M. Fiore and J. M. Barcelo-Ordinas, “Cooperative download in urban vehicular networks,” in Proc. 6th IEEE Int. Conf. MASS, Macau, Oct. 12–15, 2009, pp. 20–29. [25] L. Zhou, Y. Zhang, K. Song, W. Jing, and A. V. Vasilakos, “Distributed media services in P2P-based vehicular networks,” IEEE Trans. Veh. Technol., vol. 60, no. 2, pp. 692–703, Feb. 2011. [26] J. Burgess, B. Gallagher, D. Jensen, and B. Levine, “MaxProp: Routing for vehicle based disruption-tolerant networks,” in Proc. 25th IEEE INFOCOM, Barcelona, Spain, Apr. 23–29, 2006, pp. 1–11. [27] L. Zhou et al., “Cross-layer rate control, medium access control and routing design in cooperative VANET,” Comput. Commun., vol. 31, no. 12, pp. 2870–2882, Jul. 2008. [28] I. Leontiadis and C. Mascolo, “GeOpps: Geographical opportunistic routing for vehicular networks,” in Proc. IEEE Int. Symp. WoWMoM, Espoo, Finland, Jun. 18–21, 2007, pp. 1–6. [29] M. Zhang and R. S. Wolff, “A border node based routing protocol for partially connected vehicular ad hoc networks,” J. Commun., vol. 5, no. 2, pp. 130–143, Feb. 2010. [30] P. V. Krishna, S. Misra, V. Saritha, H. Agarwal, and N. Chilamkurti, “Learning automata-based virtual backoff algorithm for efficient medium access in vehicular ad hoc networks,” J. Syst. Architecture–Embedded Syst. Des., vol. 59, pt. B, no. 10, pp. 968–975, Nov. 2013. [31] S. Misra et al., “A fault-tolerant routing protocol for dynamic autonomous unmanned vehicular networks,” in Proc. IEEE ICC, 2013, pp. 3525–3529. [32] P. B. F. Duarte, Z. M. Fadlullah, A. V. Vasilakos, and N. Kato, “On the partially overlapped channel assignment on wireless mesh network backbone: A game theoretic approach,” IEEE J. Sel. Areas Commun., vol. 30, no. 1, pp. 119–127, Jan. 2012. [33] A. V. Vasilakos and G. I. Papadimitriou, “A new approach to the design of reinforcement schemes for learning automata: Stochastic estimator learning algorithm,” Neurocomputing, vol. 7, no. 3, pp. 275–297, Apr. 1995. [34] J. Torkestani and M. Meybodi, “Mobility-based multicast routing algorithm for wireless mobile ad-hoc networks: A learning automata approach,” Comput. Commun., vol. 33, no. 6, pp. 721–735, Apr. 2010. [35] S. Misra, V. Krishna, and V. Saritha, “LACAV: An energy efficient channel assignment mechanism for vehicular ad hoc networks,” J. Supercomput., vol. 62, no. 3, pp. 1241–1262, Dec. 2012. [36] N. Kumar and J. Kim, “ELACCA: Efficient learning automata based cell clustering algorithm for wireless sensor networks,” Wireless Pers. Commun., vol. 73, no. 4, pp. 1495–1512, Dec. 2013. [37] N. Kumar, N. Chilamkurti, and J. P. C. Rodrigues, “Learning automatabased opportunistic data aggregation and forwarding scheme for alert generation in vehicular ad hoc networks,” Comput. Commun., vol. 39, pp. 22–32, Feb. 2014. [38] N. Kumar and N. Chilamkurti, “Collaborative trust aware intelligent intrusion detection in VANETs,” Comput. Elect. Eng., vol. 40, no. 6, pp. 1981– 1996, Aug. 2014. [39] M. Esnaashari and M. R. Meybodi, “A cellular learning automata based deployment strategy for mobile wireless sensor networks,” J. Parallel Distrib. Comput., vol. 71, no. 7, pp. 988–1001, Jul. 2011. [40] S. Yousefi, E. Altman, R. El-Azouzi, and M. Fathy, “Analytical model for connectivity in vehicular ad hoc networks,” IEEE Trans. Veh. Technol., vol. 57, no. 6, pp. 3341–3356, Nov. 2008. [41] R. Lu, X. Lin, H. Zhu, and X. Shen, “An intelligent secure and privacypreserving parking scheme through vehicular communications,” IEEE Trans. Veh. Technol., vol. 59, no. 6, pp. 2772–2785, Jul. 2010. [42] J. Härri, F. Filali, C. Bonnet, and M. Fiore, “VanetMobiSim: Generating realistic mobility patterns for VANETs,” in Proc. 3rd Int. Workshop VANET, Los Angles, CA, USA, Sep. 29, 2006, pp. 96–97. [43] T. Camp, J. Boleng, and V. Davies, “A survey of mobility models for ad hoc network research,” Wireless Commun. Mobility Comput., vol. 2, no. 5, pp. 483–502, Aug. 2002. [44] A. Vasilakos, M. P. Saltouros, A. F. Atlassis, and W. Pedrycz, “Achieving QoS in hierarchical ATM networks using computational intelligence techniques,” IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 33, no. 3, pp. 297–312, Aug. 2003.

Authors’ photographs and biographies not available at the time of publication.