Clock Synchronization in Wireless LANs without Hardware Support. Aneeq Mahmood, Georg Gaderer, and Patrick Loschmidt. Austrian Academy of Sciences.
Clock Synchronization in Wireless LANs without Hardware Support Aneeq Mahmood, Georg Gaderer, and Patrick Loschmidt Austrian Academy of Sciences Viktor Kaplan Straße 2, Wiener Neustadt, A-2700, Austria. {Aneeq.Mahmood, Georg.Gaderer, Patrick.Loschmidt}@oeaw.ac.at
Abstract One typical approach to provide industrial automation networks with real-time guarantees is to plan a communication timeframe, which relies on synchronized clocks at all communication participants. This paper investigates how to reach such a clock synchronization accuracy in a wireless LAN, where commercially available chipsets dictate the implementation possibilities. Furthermore, this paper proposes a synchronization scheme which is in its implementation – an addendum to the beacons – loadindependent1 .
1. Introduction Industrial applications usually utilise wired networks for their automation tasks. The advantages are obviously the reliability of the physical communication channel as well as the exclusive access to the medium. This advantage is usually taken in spite of the possibilities of wireless communication which offers its own set of pros such as mobility and absence of hard-wired connections between the nodes and the programmable logic controller (PLC). This not only voids the cabling costs but also offers a new degree of freedom and ease of reconfiguration for applications. Besides the use of IEEE 802.15.4 and Bluetooth, which are perfectly suited for a cable replacement, other more complex applications require the introduction of networking concepts in the system. This is because many communication relationships can co-exist in these applications which requires system-wide frequency planning and handover management. IEEE 802.11 [?] wireless LAN is a suitable candidate for these applications. In order to cope with the need of being able to support several real-time communication relationships within one shared media network (such as IEEE 802.11 WLAN), several approaches exist. Among those, one is to use synchronized clocks for a time-slotted channel arbitration scheme. This time-triggered approach relies on provisioning of a 1 The work presented in this paper is founded by the FIT-IT Project 𝜀-WiFi Embedded Position Determination and Security in Wireless Fidelity Networks. Grant Number 813310, the EU Project flex WARE under grant number 224350 as well as the province of Lower Austria, the European Regional Development Fund and Oregano Systems GmbH.
synchronized clock to all network participants. With the help of synchronized clocks, these systems then plan realtime communication using time slots. These latter are filled with the real-time traffic, whose pattern is typically known in advance in terms of periodicity, data-length, and deadline requirements. This very generic scheme, however, relies on the exact planning of the available time slots and the guarantees of these timely constraints. The efficiency of this time-slicing approach is however directly dependent on the reachable clock synchronization accuracy. This paper describes an ongoing work in highprecision clock synchronization for WLAN and will provide some preliminary results for synchronization without special hardware requirements. The remainder of this paper will first continue the discussion of the problem and then describe a prototypical implementation. Finally, results together with some conclusions will be presented.
2. Problem Statement and Possible Solutions The main problem of clock synchronization is to steer all clocks need in a way that they show at every moment the same time. Second, it is important to mention that the synchronization has to be independent of the load of the network and must not create any (recognisable or otherwise) overhead. This requirement has the background that even in a fully loaded network the synchronization should keep the clock in the specified accuracy range. Beacons are an integral part of WLANs as they provide network management information and timestamps for synchronization. However, these timestamps are meant only for synchronizing WLAN cards to coordinate activities and not for the upper layers. With the current breed of open source WLAN drivers, it is possible to modify the beacons to include system level timestamps which can then be used to synchronize the whole system without providing extra synchronization traffic. Another advantage of using beacons for this purpose is that as they are highest priority packets in WLAN, which are sent periodically and do not suffer from random delays inside the transmission queues in the WLAN device. The clear disadvantage is that the extension of beacons is not standard compliant and thus only modified WLAN stacks can be used. Therefore, for this paper, it has been decided to
PPS
SYNC Packet App.
Packet Receiver
UDP/IP
UDP/IP
mac80211 + dev. driver
SYNC Packet
System Clock
Tx. timestamp
SYNC Packet
USER SPACE KERNEL SPACE
Tx. Timestamp
HARDWARE
AP
Timestamp Accum.
Tx. timestamp
Timestamp Accum.
Offset Clock Servo
Rx. Timestamp
mac80211 + dev. driver
MAC (Firmware)
MAC (Firmware)
Wireless Interface
Wireless Interface
System Clock
Clock Update
STA PPS
Oscilloscope
Figure 1. Implementation concept and data flow for clock synchronization support over WLAN use standard frames for carrying synchronization information and thus provide a degree of synchronization in the network. Through experimentation it is shown that such a scheme will work fine but will result in poor performance when it has to deal with contentions on channel access because of carrier sense multiple access/ collision avoidance (CSMA/CA) scheme in the presence of other devices, offering periodic load, in WLAN. The problem of employing beacons has been presented by Mock et al. [?] as a part of a synchronization scheme over WLAN. However, the accuracy achieved is shown to be 200 µs which is not enough for most industrial applications. Further, the authors based their work on proprietary WLAN drivers which may make such an implementation problematic in terms of costs for maintenance in industrial applications. The idea of synchronization over WLAN has also been investigated in [?], where it is suggested that microsecond range is possible over WLAN with COTS hardware without showing an actual implementation. In [?] a software-based time division media access scheme is provided by using the Timing Synchronization Function (TSF) of WLAN chipset. The system clocks of master and slave synchronize with the TSF and provide a microsecond range accuracy. This scheme, however, is primarily meant for multimedia applications. Synchronization related information, such as the gathering of timestamps, is not described. The IEEE 802.11v task group, which deals with network management, is also working towards a solution where the beacons will carry UTC offset thus establishing a common notion of time over the whole network. However, this work is still under progress and patents governing the hardware design of this next generation of WLAN cards are in a phase of acceptance.
is that they use widely available WLAN chipsets and the manipulation of the wireless LAN communication stack is separated from the development system. Thus, one only needs to update an embedded system, rather than having to restart drivers in a host system during development which is always problematic. 3.1. Basic Clock Synchronization The main idea of this work is to provide schemes for synchronizing system clocks at the application layer. For that, one can use the MAC Layer Management Entity (MLME) higher layer synchronization (MLME-HL-SYNC) primitive of the IEEE 802.11 standard [?]. With this primitive, synchronization packets carrying timestamps can be sent from the application layer of the AP, whose clock is the master, to the STA whose clock acts as a slave. The scheme is shown in figure 1. As shown in this figure, the AP sends a synchronization SYNC packet with a higher layer timestamp and an index number in it. The slave makes a timestamp when the packet arrives in the kernel space, having traversed through the PHY and the MAC layer, and calls it ‘Rx.timestamp’ as highlighted in figure 1. After the synchronization packet, the master sends a follow up packet which contains the lower layer timestamp, gathered with the help of the device driver in kernel space, of the instant when the previous packet actually has left the AP and denotes the timestamp as ‘Tx.timestamp’. This packet is also received at the STA and is forwarded to the synchronization application on the receiver side. The Rx.timestamp and Tx.timestamp are then used by timestamp accumulator at the receiver to generate the clock offset. This offset is handed over to the clock servo which uses it to control the system clock. The communication delay from AP to STA is calculated a posteriori and is also passed on to the accumulator on the receiver side to calculate the correct offset. The final accuracy is determined by generating a one Pulse Per Second (PPS) signal from the respective system clocks of STA and AP and comparing them on an oscilloscope.
3. Platform and Implementation Setup To carry out synchronization activities in this study, two firmware development boards are used which have a 533 MHz processor each, along with external memory, debug interface, and other peripheral devices. One of these boards, acting as the access point (AP), provides the master clock while the other, acting as the client (STA), comes up with the slave clock. The advantage using these boards
3.2. Modification of Lower Layer WLAN Services One practical problem for the proof of concept presented in this paper is that it is not straight forward to mod2
ify the lower layers of wireless LAN. Reasons for that are the usually fully firmware-integrated Media Access Controller (MAC) functionality in chipsets and the poor documentation provided by the stakeholders of the wireless LAN mass market. One possibility for modifying existing stacks – rather than re-programming a whole stack from the beginning – is exploitation of open source Linux device drivers for COTS WLAN cards. Among those ath5k [?] is such a driver, based on mac80211 [?], which is developed and maintained by the open source community. These device drivers perform their tasks on top of the MAC layer whereas the actual media access controller is implemented in firmware. However, to suppress the size of this firmware, the ath5k driver supports functionalities which enable the deployment of MLME in the Linux user space. This eases network management as operations such as beacon generation and handling can be controlled more easily from the user space than from kernel space.
timestamps from the master to the slave every second. In this way, the four timestamps are filtered by a nonlinear filter. In particular this filter removes outliers by a minmax search and averages the remaining samples. Nevertheless, other sources of jitter for timestamping still remain in the system which include the jitter in the PHY and MAC layer, especially during the packet reception. This jitter along with the jitter because of process scheduling inside the kernel has to be found out to analyse the accuracy of the timestamps. 3.4. Measurement Setup In order to investigate the synchronization accuracy over the wireless channel and its behaviour with external traffic, a set up has been created which consists of two WLAN devices in the form of embedded systems running OpenWrt [?]. OpenWrt provides application layer synchronization packets which are timestamped in the kernel space. Further, a simple control loop is run at the slave side to adjust the system clock for clock drift and offset. The entities of the WLAN are set close to each to ensure high SNR and minimisation of the jitter in the PHY. For measuring the synchronization accuracy, two systems (master and the slave) have been configured to provide 1 PPS from the system clock. This is done by generating a rising edge on a pin of the serial port. By measuring the delay between those two pulses, the inaccuracy of the slave clock can be determined. Figure 2 shows the synchronization of an idle system.
3.3. Highly Accurate Timestamping One general rule for clock synchronization is that the quality of the obtained synchronization is directly influenced by the precision (jitter) of the timestamps. In order to improve the quality of timestamps, one tries typically to draw them at the lowest possible OSI layer and ideally at the physical layer (PHY). For WLAN implementations this point is hard to identify, as chipsets nowadays typically define only a very vague border between PHY and MAC. One possible point of timestamping can be near the radio frequency to baseband conversion section inside the chipsets as mentioned in [?]. However, this kind of timestamping is not possible for COTS WLAN cards, as it would require hardware modifications to get access to the PHY and MAC (firmware in the chipset) layers. Thus, timestamping has to be done by software means inside the device driver as shown in figure 1. The timestamping instant is chosen whenever a packet is completely sent or received. When a packet is sent or received, the hardware notifies the driver with a hardware interrupt. The driver masks the interrupt, takes the timestamp using the getnstimeofday interface of the kernel and then instigates a corresponding interrupt handling routine in the form of kernel tasklets. In the corresponding tasklets, the packet index number and the timestamps are analysed and then are handed to application layer as shown in figure 1. However, when software timestamping is carried out, jitter is introduced which can degrade the synchronization results. For example, packet transmission is actually carried out in hardware and the driver is notified of a successful transmission through an interrupt. The context switching of the processor and the operating system commonly introduces jitter to the timestamps. Similar jitter is systematically introduced at the reception. The combined jitter can degrade the overall synchronization accuracy. To minimise this jitter, packet filtering is carried out by choosing a synchronization interval of 4 s while sending
350
No. of Observations
300
Mean= −397 ns Std. dev = 3.17 µs
250 200 150 100 50 0 −2
−1.5
−1
−0.5 0 0.5 Time Offset (s)
1
1.5
2 −5
x 10
Figure 2. Synchronization accuracy between AP and STA without payload transmission The second experiment conducted for this paper is done by investigating the loaded behaviour. To create the notion of traffic load, one additional node (1.8 GHz PC) is added to the system which does not use the synchronization packets but will only generate a regular traffic on the network whose period resembles to that of cycle times. This behaviour is not so different from actual implemen3
WLAN using COTS devices with high-resolution timer support from the open source community and highlights the impact of external traffic on synchronization accuracy. In principle, if synchronization messages can be sent out on a strict timeframe such as the beacon interval, it has several advantages as synchronization is maintained without additional system load. On the other hand, if synchronization packets are attached to beacons, interoperability with standard systems cannot be maintained anymore. For these investigations a solution has been chosen where the synchronization packets are embedded directly into standard frames, but a sophisticated software design for accurate timestamping is chosen. This leads to the result that the increase of traffic load corresponds to a worse accuracy. The situation is similar to a higher synchronization interval than actual because of increasing contentions on the channel, leading to a drop in synchronization accuracy. Thus, the idea of using beacon packets to carry out synchronization information is advocated as they present no extra traffic. The future work includes finding out the limits of synchronization accuracy when beacons are modified to carry the timestamps and a more comprehensive analysis on the overall impact of traffic load on synchronization accuracy along with a complete delay profile of timestamping and clock offsets.
500 Mean= −529 ns Std. dev = 5.27 µs
No. of Observations
400
300
200
100
0 −3
−2
−1
0 Time Offset (s)
1
2
3 −5
x 10
Figure 3. Synchronization accuracy between AP and STA with traffic on the channel
tation because the synchronization packets are broadcast packets anyway and no other WLAN device can talk on the channel while it is used by the other. Thus, if one sends a packet for synchronization to only one slave or broadcasts it, it does not make any difference in the overall scheme of things. The data rate on the channel has been fixed to 1 Mbps to increase the robustness of the connection. The load is generated from another STA by using ICMP packets but only in uplink. The AP only receives the packets and does not send back any replies except for ACK packets. These packets are sent periodically with a period of 500 ms and payload consists of 1024 bytes.
References [1] “IEEE Standard 802.11 - wireless LAN medium access control (MAC) and physical layer (PHY) specifications. The Institute of Electrical and Electronics Engineers, Inc., 2007”. [2] M. Mock, R. Frings, E. Nett, and S. Trikaliotis, “Clock synchronization for wireless local area networks”, in Proc. 12th Euromicro Conference on Real-Time Systems Euromicro RTS 2000, 2000, pp. 183–189. [3] T. Cooklev, J. Eidson, and A. Pakdaman, “An Implementation of IEEE 1588 Over IEEE 802.11b for Synchronization of Wireless Local Area Network Nodes”, IEEE Trans. Instrum. Meas., vol. 56, no. 5, pp. 1632–1639, 2007. [4] F. Guo and T. Chiueh, “Comparison of QoS guarantee techniques for VoIP over IEEE802.11 wireless LAN”, in Proc. of 15th Annual Multimedia Computing and Networking Conference (MMCN 2008), January 2008. [5] ath5k, “the madwifi project”, http:// madwifi-project.org/wiki/About/ath5k Last accessed: 28/02/2010. [6] J. W. Linville, “Tux on the Air: The State of Linux Wireless Networking”, in Proceedings of the Linux Symposium, volume 2, July 2008, pp. 39–46, Ottawa, Ontario, Canada. [7] R. Exel, J. Mad, G. Gaderer, and P. Loschmidt, “A Novel, High-Precision Timestamping Platform for Wireless Networks”, in ETFA’09: The 14th International Conference on Emerging Technologies and Factory Automation, Sept. 2009, pp. 1–8. [8] OpenWrt, “Openwrt, Linux distribution for embedded devices”, http://openwrt.org Last accessed: 28/02/2010.
4. Results and Conclusions Figure 2 presents the synchronization accuracy in the form of clock offset for the case when there exists nominal traffic on the channel and the only meaningful activity is between the STA and the AP exchanging synchronization packets. The offset in this case is found to be −397 ns and the standard deviation is 3.17 µs. However, in the presence of a further node in the network, the offset is found to be −529 ns and a standard deviation of 5.27 µs as indicated in figure 3. For each of these trends 4000 samples have been collected. In case the synchronization interval is increased to 8 s with nominal load, the clock offset is found out to be −0.6 µs and a standard deviation of 15.2 µs. This indicates that the situation when the load on the system increases begins to resemble as that of using a higher synchronization interval as the synchronization accuracy drops. Hence, when the synchronization information is provided by additional traffic on the network, the load increases and increases the contention on the channel. Consequently, the synchronization interval increases and overall accuracy decreases. This study presents the synchronization accuracy in 4
