2nd IEEE Workshop on Multimedia Communications & Services
Testbed-based Performance Evaluation of Routing Protocols for Vehicular Delay-Tolerant Networks João A. Dias1, João N. Isento1, Vasco N. G. J. Soares1,2, Farid Farahmand3, and Joel J. P. C. Rodrigues1 1
Instituto de Telecomunicações, University of Beira Interior, Covilhã, Portugal 2 Polytechnic Institute of Castelo Branco, Portugal 3 Sonoma State University, CA, USA.
{joao.dias, joao.isento}@it.ubi.pt,
[email protected],
[email protected],
[email protected] Abstract— As an application of the concept of delay-tolerant network (DTN) for vehicular communications, the vehicular delay-tolerant network (VDTN) architecture was proposed to cope with issues, such as highly dynamic network topology, short contact durations, disruption, intermittent connectivity, variable node density, and frequent network fragmentation. These challenging characteristics of vehicular networks affect the design and performance of routing protocols. This paper presents a testbed performance evaluation of DTN-based routing protocols applied to VDTNs. The objective is to evaluate and understand how popular routing strategies perform in sparse or partitioned opportunistic vehicular network scenarios. It was observed that Spray and Wait routing protocol outperforms all other protocols considered in the study.
from the perspective that vehicular networks are disconnected by default. To address these issues, researchers have exploited the store-carry-and-forward model of routing proposed for Delay Tolerant Networks (DTNs) [3]. The idea behind it is to exploit the physical motion of vehicles and opportunistic contacts to transport data between disconnected parts of the network. Such approach overcomes network partitions, allowing delay-tolerant data traffic from a variety of vehicular applications to be routed over time [4-7]. The Vehicular Delay-Tolerant Network (VDTN) architecture [8] is a layered architecture that employs a store-carry-and-forward approach to achieve reliable communications in vehicular environments. VDTN architecture differs from other proposals available in the literature in several respects. It combines an IP over VDTN approach with control plane and data plane separation, and out-of-band signaling. The main idea of VDTN architecture is to assemble IP packets into variable length data bundles, and transmit these data bundles asynchronously via a data plane connection. The data plane connection is set up using out-ofband signaling information previously transmitted through a separate control plane connection. In this work, a VDTN testbed [9] is used for conducting experiments to evaluate and perform a comparison study of distinct DTN-based routing strategies applied to vehicular delay-tolerant networks. The protocols are evaluated in terms of bundle delivery probability, bundle average delivery delay, and number of dropped bundles, in different network scenarios. The remainder of the paper is organized as follows. Section II presents related work focusing on DTN routing protocols. Section III describes the testbed and the performed experiments. The performance evaluation of the routing protocols may be found in Section IV and Section V provides the main conclusions and directions for future work.
Keywords— Vehicular Delay-Tolerant Networks; Performance Evaluation; Store-carry-and-forward; Routing; Testbed
I. INTRODUCTION Vehicular networks have been the focus of an increasing interest in the research community in the last few years. In part, this interest is due to the potential applications of these networks that include, but are not limited to, road safety, driving assistance, road traffic optimization, monitoring, and a wide variety of commercial and entertainment applications. They can also be used to provide connectivity to remote/rural communities and regions, and support for communication between rescue teams in catastrophe-hit areas. The characteristics of vehicular networks pose unique challenges to the design of routing protocols [1]. In particular, they have a highly dynamic network topology, variable node density, and are characterized by short contact durations. Limited transmission ranges, radio obstacles, and interferences, make these networks prone to disruption, intermittent connectivity, and significant loss rates. Because of these issues, vehicular networks experience frequent partition (i.e., end-to-end connectivity may not exist). A variety of factors including node heterogeneity, node interactions, node cooperation and limited network resources, also pose additional challenges. Routing protocols designed for fully connected networks are not suitable for data delivery in sparse, intermittent, partially connected, opportunistic vehicular networks [2]. Hence, there has been a need to design routing techniques
978-1-4673-0040-7/11/$26.00 ©2011 IEEE
II. RELATED WORK Vehicular delay-tolerant networks rely on opportunistic contacts between network nodes to deliver data in a storecarry-and-forward DTN paradigm that works as follows. A source node originates a data bundle and stores it using some form of persistent storage, until a communication opportunity (i.e., a contact) arises. This bundle may be forwarded when the
591
source node is in contact with an intermediate node that can help bundle delivery. Afterwards, the intermediate node stores the bundle and carries it until a suitable contact opportunity occurs. This process is repeated and the bundle will be relayed hop by hop until reaching its destination (eventually and over time). Figure 1 illustrates this concept.
First Contact [10] performs routing by forwarding bundles randomly. Nodes forward bundles to the first node they encounter. This results in a random search for the destination node. Moreover, bundles may oscillate among a set of nodes, or be delivered to a dead end. Examples of well-known multiple-copy routing protocols include Epidemic [14], Spray and Wait [15], and PRoPHET [16]. These protocols make different assumptions about the knowledge available to network nodes (e.g. absence of knowledge, history of node encounters), as discussed below. Epidemic does not require any prior knowledge about the network. In this routing protocol, bundles are replicated to all encountered nodes. Epidemic suffers from the disadvantages of flooding as the node density increases. In an environment with infinite buffer resources and bandwidth, this protocol provides an optimal solution, since it delivers all the bundles that can possibly be delivered in the minimum amount of time. Spray and Wait [15] limits the number of bundle copies created per bundle in order to control flooding. Bundle copies are initially sprayed (i.e., distributed) to nodes until the number of copies is depleted (in this study is assumed that this number is 3). Two spraying schemes are proposed in [15]. In the source spray scheme, the source node forwards one of the copies to each encountered node. In the binary spray scheme, half of the bundle copies are forwarded to each encountered node. If the destination node is not found during the “spray phase”, then at the “wait phase” direct transmission is performed (i.e. the bundle copy left is forwarded only to its destination). PRoPHET [16] is a probabilistic routing protocol. It considers the history of node encounters and transitivity information to calculate probabilistic metric called delivery predictability. This metric is used to decide whether or not to forward bundles at contact opportunities. A bundle is forwarded to another node if the delivery predictability of the destination of the bundle is higher at that node.
Fig. 1. Illustration of the store-carry-and-forward paradigm with the three VDTN different type of nodes (Terminal, Relay , and Mobile nodes).
Routing in any DTN-based network, like a VDTN, can be defined as a sequence of independent, local forwarding/replication decisions that make bundles “progress in steps” towards their destination. The source of knowledge that is used to take these decisions often differs. While some routing approaches assume that there is not any knowledge available, others consider and eventually combine information about historical data (e.g., recent encounters, contact time, contact frequency, or contact location), location (e.g., past, present, future location data), or movement patterns [10]. DTN-based routing strategies can also be classified based on the number of copies of bundles disseminated through the network [11]. Single-copy schemes maintain a single copy of a bundle in the network that is forwarded between network nodes. On the contrary, multiple-copy schemes replicate bundles at contact opportunities. Bundle replication improves the probability of delivery and minimizes the delivery latency. Nevertheless, it consumes a high amount of energy, and increases the contention for network resources like bandwidth and storage. Therefore, it can lead to poor overall network performance, as discussed in [11, 12]. The following are examples of single-copy and multiplecopy routing protocols, which are aimed at generic application scenarios. Thus, they can be potentially used in any DTNbased network, such as a VDTN and, for this reason, are considered in this work. Direct Delivery/transmission [13] and First Contact [10] are two examples of simple single-copy routing protocols, which do not use any knowledge about the network to make forwarding decisions. In Direct Delivery [13], the node carries bundles until it meets the destination node(s). Thus, although this protocol has minimal overhead, it may incur in very long delays for bundle delivery.
III. TESTBED EXPERIMENTS This section describes VDTN@Lab, a laboratory testbed for VDTN networks, which aims to provide a framework for the validation and performance evaluation of the VDTN architecture, protocols, services, and applications. Next subsections present details regarding the testbed and the network scenarios considered in the experiments. A. Testbed Specifications In VDTN networks both fixed and mobiles nodes can freely interact with each other. Three node types compose the VDTN architecture. Terminal nodes are located in isolated regions acting as access points to the VDTN network, representing the network edge. Mobile nodes move on roads, carrying data between terminal nodes. They are exploited to collect and disseminate data bundles through the VDTN network. Stationary relay nodes are fixed devices located at road intersections. They have store-and-forward capabilities in order to allow mobile nodes to pickup and deposit data. In a
592
MB. These buffers are managed according to a FIFO (“drop head”) policy. Data bundles are generated with a time interval of 20 seconds by random mobile nodes and are destined to random terminal nodes. Two different scenarios are considered in this study. In the first one (Scenario 1), the bundle size changes between 128, 256, 512, 1024, and 2048 Kbytes across experiments and the bundles’ TTL is fixed to 20 minutes. In Scenario 2, the bundles’ TTL changes between 5, 10, 15, and 20 minutes across experiments. In this scenario, the bundles’ size is uniformly distributed between 256 Kbytes and 2 Mbytes. Each testbed experiment runs during an hour and it is assumed a fully cooperative opportunistic environment. The performance metrics considered in this study are the bundle delivery probability, the bundle average delay, and the number of dropped bundles. The bundle delivery probability is measured as the relation between the number of unique delivered bundles and the number of bundles sent. The bundle average delay is measured as the time between bundles creation and delivery. Finally, the number of dropped bundles reports the number of bundles that have been discarded from the nodes’ buffers due to buffer overflow or TTL expiration.
scenario with low node density, relay nodes increase the number of contact opportunities. Figure 2 illustrates the interactions between VDTN network nodes.
Fig. 2. Illustration of a VDTN network scenario.
VDTN@Lab was created to demonstrate the VDTN architecture and its services, protocols, and applications. It consists in desktops/laptops, netbooks, and robots. Desktops and laptops are used to emulate terminal nodes and relay nodes. Mobile nodes are emulated using LEGO MINDSTORMS NXT robots with coupled netbooks. These netbooks provide networking and storage capabilities. LEGO NXT robots are programmed for having a random movement across roads. All of these network nodes support Bluetooth and IEEE 802.11b/g technologies that are used to enable out-of-band signaling with the separation of control and data planes proposed in VDTN architecture [8]. Bluetooth is used to exchange signaling information, and IEEE 802.11g is used to exchange data bundles. Software modules were created in C# programming language and deployed in the network nodes to emulate the VDTN protocols, algorithms, services and applications. They were developed using the .NET Framework for running in laptops and desktops with Microsoft Windows 7 operating system. In addition, the software modules also provide management tools and advanced statistics reports.
Fig. 3. Photos of the VDTN@Lab testbed running the experiments.
IV. PERFORMANCE EVALUATION This section presents the performance results for the testbed experiments described above. Two scenarios are considered and associated results are presented in the next subsections.
B. Network Scenarios The scenario that was set up to conduct experiments on the VDTN testbed has a dimension of 36,5m2 and is shown in Figure 3. It consists in three terminal nodes, two relay nodes, and four mobile nodes. Terminal nodes are placed at different points of the laboratory. Stationary relay nodes are placed on road intersections. Taking into account a study presented at [17], and assuming a scale of 1:50 (1m in the laboratory testbed represents 50m in a real scenario), mobile nodes move with speeds of 48 Km/h, 40 Km/h, 36 Km/h, and 24 Km/h. The nodes’ buffers have different capacities according to their roles in the network. Terminal nodes have a buffer with a 50 MB of capacity, relay nodes 75 MB, and mobile nodes 25
A. Performance Analysis for Scenario 1 This study starts with the performance evaluation of the effect of the bundle size on different routing protocols. Due to the limited bandwidth and short contact duration, only a limited number of bundles can be transferred at a contact opportunity. Hence, as may be seen in Figure 4 (a), the bundle size has a direct influence on the delivery probability of all routing protocols. As expected, one can observe that multiple-copy routing protocols present better delivery ratios than single-copy protocols. In particular, Spray and Wait presents the best
593
Delivery Probability (%)
First Contact Direct Delivery Epidemic Spray and Wait PRoPHET
128
256
512
1024
First Contact
600
Direct Delivery
500 400
Epidemic
300
Spray and Wait
200 100
PRoPHET
(c)
800
First Contact
700 Direct Delivery
600 500
Epidemic
400 300
Spray and Wait
200 100
PRoPHET
0
0 128
2048
900
(b)
700
Number of Dropped Bundles
(a)
Bundle Average Delay (seconds)
800
100 90 80 70 60 50 40 30 20 10 0
256
512
1024
128
2048
Bundles Size (KB)
Bundles Size (KB)
256
512
1024
2048
Bundles Size (KB)
Delivery Probability (%)
(a)
First Contact Direct Delivery Epidemic Spray and Wait PRoPHET
5
10
15
Bundles Time-to-Live (minutes)
20
900
800
(b)
700
First Contact
600
Direct Delivery
500 400
Epidemic
300 Spray and Wait
200 100
PRoPHET
0 5
10
15
Bundles Time-to-Live (minutes)
20
Number of Dropped Bundles
100 90 80 70 60 50 40 30 20 10 0
Bundles Average Delay (seconds)
Fig.4. Bundle Delivery probability (a); Bundle average delay (b); and Number of dropped bundles (c) as function of bundle size for First Contact, Direct Delivery, Epidemic, Spray and Wait, and PRoPHET routing protocols. (c)
800
First Contact
700 Direct Delivery
600 500
Epidemic
400 300
Spray and Wait
200 100
PRoPHET
0 5
10
15
20
Bundles Time-to-Live (minutes)
Fig.5. Bundle Delivery probability (a); Bundle average delay (b); and Number of dropped bundles (c) as function of bundle TTL for First Contact, Direct Delivery, Epidemic, Spray and Wait, and PRoPHET routing protocols.
B. Performance Analysis for Scenario 2
results across all the experiments. It improves the delivery probability approximately in 3%, 7%, 6%, 12%, and 8% (for each of the considered values of bundles size) when compared to Epidemic, and 22%, 28%, 32%, 35%, and 28% when compared to PRoPHET. The performance differences between Spray and Wait and Epidemic would be more pronounced with more restrictions on bandwidth and storage space. Figure 4 (b) shows that, besides improving the delivery probability, Spray and Wait also presents the lowest average delivery delays. When compared to Epidemic, bundles arrive to the destination approximately 16, 18, 66, 98, and 175 seconds sooner in average, and approximately 69, 76, 114, 145, and 215 seconds sooner when compared to PRoPHET. From Figure 4 (b) it is also possible to see that single-copy routing schemes suffer from longer average delivery delays. For example, when Spray and Wait is compared with Direct Delivery, bundles arrive at the destination approximately 108, 116, 136, 185, and 268 seconds sooner on average. As a final note to this Figure, it can be seen that lower average delays are obtained for smaller bundle sizes. Figure 4 (c) displays the results regarding the number of dropped bundles. Due to its pure flooding scheme, Epidemic presents the worst results. Spray and Wait succeeds in limiting replication by setting an upper bound on the number of replicas created per bundle. Thus, saving network resources (e.g., bandwidth and storage). Since single-copy routing strategies maintain only one copy of each bundle in the network. Thus, as expected, First Contact and Direct Delivery register the lowest numbers of dropped bundles.
The second scenario evaluates how the performance of routing protocols is affected by the bundles’ TTL value. Intuitively, increasing the TTL value leads to having more bundles stored at the network nodes’ buffers during longer periods of time. As may be observed in Figure 5 (a), this increases the probability of bundle delivery. Like in the previous scenario, the results depicted in Figure 5 (a) confirm that Spray and Wait routing protocol performs better than the other protocols tested in this study. Compared to Epidemic, Spray and Wait increases the delivery probability in about 14%, 8%, 10%, and 9% for each of the considered values of TTL, respectively. When compared to PRoPHET, it presents gains of 22%, 21%, 22%, and 19% in this performance metric. Direct Delivery and First Contact single-copy routing strategies have the worst delivery probabilities again, for the same reasons stated in the previous scenario. Maintaining a single copy of each bundle results in frequently dropping bundles before they arrive at the destination terminal node. This Figure shows that Direct Delivery performs better than First Contact. Nevertheless, it is interesting to observe that compared to First Contact, Spray and Wait increases the delivery probability in 41%, 45%, 44%, and 41%. From the combined analysis of Figure 5 (a) and (b), it is possible to observe that increasing the bundles’ TLL leads to improved delivery ratios but also to longer average delivery delays. Again, the Spray and Wait routing protocol provides lower delivery delays for all the tested TTL values. Note that the delivery delay difference between this protocol and the other protocols increases with larger TTLs. In this scenario, Spray and Wait decreases the bundles average delivery delay in approximately 13, 70, 149, and 233
594
seconds when compared to Epidemic, and 98, 199, 300, and 404 seconds compared to First Contact. Figure 5 (c) confirms the observations obtained in the first scenario. Epidemic presents the worst results regarding the number of dropped bundles. On the contrary, Direct Delivery registers the lowest numbers of dropped bundles across the experiments.
[4]
Y. Shao, C. Liu, and J. Wu, "Delay-Tolerant Networks in VANETs," in Vehicular Networks: From Theory to Practice, S. Olariu and M. C. Weigle, Eds.: Chapman & Hall, 2009.
[5]
L. Franck and F. Gil-Castineira, "Using Delay Tolerant Networks for Car2Car Communications," in IEEE Int. Symp. on Industrial Electronics 2007 (ISIE 2007), Vigo, Spain, 4-7 June, 2007, pp. 2573-2578.
[6]
J. Burgess, B. Gallagher, D. Jensen, and B. Levine, "MaxProp: Routing for Vehicle-Based Disruption-Tolerant Networks," in 25th IEEE
V. CONCLUSIONS AND FUTURE WORK
International Conference on Computer Communications (INFOCOM
This paper evaluated the influence of different DTN-based routing approaches on the performance of Vehicular DelayTolerant Networks. A laboratory testbed was used to conduct several experiments. Two scenarios were considered for the study, one changed the bundles’ size, and the other the bundles’ TTL. The results observed show that First Contact protocol and Direct Delivery single-copy routing approaches reduce the buffer and bandwidth usage but suffer from long delivery delay and low delivery probability. Epidemic, Spray and Wait, and PRoPHET multiple-copy schemes improve these performance metrics. Furthermore, it was concluded that Spray and Wait presents the best performance results. It consumes fewer network resources than unconstrained Epidemic flooding. The performance of PRoPHET protocol was affected by the scenario under study, namely because of the limited number of network nodes and the mobility pattern of mobile nodes. The performance analysis of these routing protocols under different fragmentation/defragmentation and aggregation/ de-aggregation approaches, as well as the implementation of MaxProp routing protocol on the VDTN laboratory testbed are some of our interests for future work.
2006), Barcelona, Catalunya, Spain, April 23-29, 2006, pp. 1-11. [7]
V. Cabrera, F. J. Ros, and P. M. Ruiz, "Simulation-based Study of Common Issues in VANET Routing Protocols," in IEEE 69th Vehicular Technology Conference (VTC2009-Spring), Barcelona, Spain, April 2629, 2009.
[8]
V. N. G. J. Soares, F. Farahmand, and J. J. P. C. Rodrigues, "A Layered Architecture for Vehicular Delay-Tolerant Networks," in Fourteenth IEEE Symposium on Computers and Communications (ISCC ’ 09), Sousse, Tunisia, July 5‐8, 2009, pp. 122-127.
[9]
J. N. Isento, J. A. Dias, João c. Neves, V. N. G. J. Soares, J. J. P. C. Rodrigues, António M. D. Nogueira, and Paulo Salvador, "FTP@VDTN - A File TRansfer Application for Vehicular Delay-tolerant Networks," in Conftele 2011, Lisbon, Portugal, April 27-29, 2011.
[10] S. Jain, K. Fall, and R. Patra, "Routing in a Delay Tolerant Network," in ACM SIGCOMM 2004 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, Portland, Oregon, USA, 2004, pp. 145-158. [11] A. Balasubramanian, B. N. Levine, and A. Venkataramani, "DTN Routing as a Resource Allocation Problem," in ACM SIGCOMM 2007, Kyoto, Japan, August 27-31, 2007, pp. 373-384. [12] Z. J. Haas and T. Small, "Evaluating the Capacity of ResourceConstrained DTNs," in International Wireless Communications & Mobile Computing Conference (IWCMC 2006) - International Workshop on Delay Tolerant Mobile Networks (DTMN), Vancouver,
ACKNOWLEDGMENTS
Canada, July 3-6, 2006, pp. 545-550.
Part of this work has been supported by the Instituto de Telecomunicações, Next Generation Networks and Applications Group (NetGNA), Portugal, by the Euro-NF Network of Excellence from the Seventh Framework Programme of EU, in the framework of the Specific Joint Research Project VDTN, and by National Funding from the FCT – Fundação para a Ciência e a Tecnologia through the PEst-OE/EEI/LA0008/2011 Project.
[13] T. Spyropoulos, K. Psounis, and C. S. Raghavendra, "Single-copy
REFERENCES
[15] T. Spyropoulos, K. Psounis, and C. S. Raghavendra, "Spray and Wait:
[1]
Routing in Intermittently Connected Mobile Networks," in First IEEE Communications
on
Sensor
and
Ad
Hoc
USA, October 4-7, 2004, pp. 235- 244. Ad Hoc Networks," Duke University, Technical Report, CS-2000-06, April, 2000. An Efficient Routing Scheme for Intermittently Connected Mobile
S. Yousefi, M. S. Mousavi, and M. Fathy, "Vehicular Ad Hoc Networks
Networks," in ACM SIGCOMM 2005 - Workshop on Delay Tolerant
(VANETs): Challenges and Perspectives," in 6th International
Networking and Related Networks (WDTN-05), Philadelphia, PA, USA, August 22-26, 2005, pp. 252-259.
June 21-23, 2006, pp. 761-766.
[3]
Conference
[14] A. Vahdat and D. Becker, "Epidemic Routing for Partially-Connected
Conference on ITS Telecommunications (ITST 2006), Chengdu, China, [2]
Society
Communications and Networks (IEEE SECON 2004). Santa Clara, CA,
[16] A. Lindgren, A. Doria, E. Davies, and S. Grasic, "Probabilistic Routing
M. Zhang and R. S. Wolff, "A Border Node Based Routing Protocol for
Protocol for Intermittently Connected Networks," draft-irtf-dtnrg-
Partially Connected Vehicular Ad Hoc Networks," J. of Communications,
prophet-06,
Academy Publisher, vol. 5, no. 2, pp. 130-143, February 2010.
http://tools.ietf.org/html/draft-irtf-dtnrg-prophet-06.
V. Cerf, S. Burleigh, A. Hooke, L. Torgerson, R. Durst, K. Scott, K. April
2007,
[Online].
Available:
12,
2010,
[Online].
Available:
[17] M. Rubinstein et al., “Measuring the Capacity of In-Car to In-Car
Fall, and H. Weiss, "Delay-Tolerant Networking Architecture," RFC 4838,
July
Vehicular Networks”, IEEE Communications Magazine, vol. 47, no. 11,
http://www.rfc-
pp. 128-136, November 2009.
editor.org/rfc/rfc4838.txt.
595
