SFC - Clemson University

8 downloads 22389 Views 2MB Size Report
Due to the high cost of ISP programmers (≈ ... the number of ISP programmers is usually large. ... flow control, uses additional dedicated control lines and soft-.
SFC: A Simple Flow Control Protocol for Enabling Reliable Embedded Network Systems Reprogramming Biswajit Mazumder, Jason O. Hallstrom School of Computing Clemson University Clemson, SC 29634

{bmazumd, jasonoh}@cs.clemson.edu ABSTRACT To support reprogramming in embedded network systems (ENSs), we present SF C, a simple flow control protocol for communication between serial devices, which enables fast and reliable data transmission. The reprogramming strategy uses a boot loader, responsible for receiving the application program over a serial communication channel, storing it in the program memory space, and then initiating execution. SF C is used as the communication protocol for the boot loader. SF C uses a finite state automata to control the send and receive mechanisms, while introducing minimal protocol overhead. The automata utilizes synchronization and acknowledgment mechanisms to achieve high reliability. We first describe the boot loader design and the reprogramming strategy. We then describe the design and implementation of the SF C protocol and present experimental results to demonstrate its performance and efficacy.

Keywords Embedded network systems, flow control protocols, boot loaders, flash programming

1. INTRODUCTION Most ENS applications are “burned into” the embedded hardware before they are deployed. The development environments provided by typical desktop systems are used to develop the applications, and the compiled program is written to the program memory of the embedded device with the use of an in-system-programmer (ISP). Typical ENS deployments for disaster management, structural heath monitoring, and volcanic activity monitoring [2, 10, 12] require a large number of these devices to be programmed and deployed. Due to the high cost of ISP programmers (≈ $30–100) as compared to an individual embedded device like the TelosB/Tmote Sky, MicaZ, or SHIMMER, often called “motes” (≈ $50–100) [9], the ratio of the number of motes to the number of ISP programmers is usually large. Thus, the programming time required to set up such a deployment is

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ACM SE ’12 Copyright 2012 ACM X-XXXXX-XX-XX/XX/XX ...$10.00.

again large. Moreover, adding new functionality or making bug fixes to existing applications can also be time-consuming — more so if it is required after a ENS deployment has been completed. Reprogramming solutions that avoid the use of ISP programmers address these problems. ISP-free reprogramming strategies usually require installing a boot loader on the target device, which receives the application program. The data transfer may rely on wired or wireless communication, terminating in a serial (e.g. UART) or parallel communication device. The transferred data is either stored in external memory, such as EEPROM, or stored directly in the application flash memory space. The transmitting side in this process is typically a computing system with large amounts of RAM (and buffer) space, able to send the program data at a continuous rate. One problem with this strategy is that the application program is usually much larger in size than the mote’s available RAM. Thus, the entire application program cannot be sent to the boot loader all at once; this would overwhelm the device. Thus, the boot loader must be able to receive a smaller block of the application program, process the data, store it to flash (or external storage), and then proceed to the next block of the application program. The rate at which the boot loader receives data is typically much faster than the speed at which it can write the data to flash; the write to flash operation is a bottleneck. This necessitates a dedicated flow control mechanism to manage the inflow of program data and to allocate RAM memory with fine-grained control. Existing hardware flow control solutions, e.g. RT S/CT S and DT R/DSR, usually require out-of-band signaling and dedicated control lines. In the context of ENSs, dedicated control lines require additional microcontroller pins – often a scarce resource. Software flow control solutions, e.g., XON/XOF F , do not consume hardware resources, but provide no reliability against bit errors. Supplemental parity bits help in error detection, but provide no reliability unless an additional retransmission strategy is also implemented. Our primary objective in this paper is to describe SF C, a simple flow control protocol, in the context of a boot loader we have developed to support serial device reprogramming. SF C provides fast and reliable transmission with minimal protocol overhead. The reliability of SF C is attributed to the fact that it can recover from bit errors.

2.

RELATED WORK

The study of flow control protocols has been ongoing for decades. In software flow control, also known as XON/XOF F flow control, byte transmission occurs using standard com-

munication lines [6]. Two bytes from the ASCII character set, called XON and XOF F , are predefined to be used for flow control purposes. Sending the character XON indicates the start of data transmission, while XOF F is sent to pause it. Since this flow control strategy is stateless, loss of either the XON or XOF F byte due to unreliable transmission lines may cause the system to become unrecoverable. Software flow control also does not provide data reliability. Hardware flow control [6], also referred to as RT S/CT S flow control, uses additional dedicated control lines and software for achieving controlled data transmission. The sender sets the RT S (request to send) line to indicate that it has data to send. If the receiver intends to receive, it sets the CT S (clear to send) line to initiate data transfer. Additional DSR (data set ready) and DT R (data terminal ready) lines may also be used to control data transmissions in null modem conections. Hardware flow control is faster than software flow control as it does not introduce control character overhead; but it also suffers from the lack of data reliability. Binary Synchronous Communication (Bisync) [11], a popular communication protocol developed by IBM, uses multiple byte control sequences, from either the ASCII or EBCDIC character sets. Error checking is achieved using either a combination of vertical redundancy checks (VRCs) and longitudinal redundancy checks (LRCs), or 16-bit cyclic redundancy checks (CRCs). Bisync uses frames without numbers, and numbered acknowledgments (ACKs) to achieve reliability [13]. The sender transmits data and waits for an ACK; on a timeout, it sends an ENQ. The receiver accepts data with a correct CRC, or sends a negative acknowledgement (NAK) on the receipt of a damaged packet. On receiving an ENQ, the receiver replies with the last ACK value it sent. For a simple data communication protocol, Bisync is needlessly complex; adhering to the protocol is difficult [13]. Sliding window protocols [13] like stop-and-wait, go-backN, receiver buffering, and selective reject (i.e., receiver buffering with out-of-order retransmissions) are data transmission protocols widely used for reliable in-order packet delivery in data link layers and TCP. Sliding window protocols use sequence numbers and ACKs in packets (or frames) to achieve reliable, full duplex data transmission. These protocols are able to identify lost packets and initiate retransmissions. Flow control is achieved by slowing down ACK delivery; overall ACK delivery rate governs the sending rate. Flow control in embedded systems is significantly different, presenting unique challenges to the protocol designer due to the limited availability of computational resources. For example, the MoteStack, a state-of-the-art in-situ sensing platform, uses an AVR Atmel microcontroller (MCU) [1]. It has 4KB of RAM, 64KB of in-system, self-programmable flash program memory, and operates at 10 MHz at 3.3V. While existing software and hardware flow control protocols are easy to implement, they lack reliability. Additionally, hardware flow control mechanisms need MCU pins. The more sophisticated protocols lack simplicity of design. Sliding window protocols are robust, but require large amounts of processing and memory resources — typically lacking in embedded systems. The serial data communication protocol in TinyOS [3] is loosely based on PPP in HDLC-like framing (RFC-1662). It uses the frame structure as in PPP and supports error detection with the help of 16-bit CRCs. The protocol also uses an acknowledgment strategy. TinyOS2 [4] includes a frame-

based protocol which also uses the HDLC encoding strategy. The protocol in TinyOS2 consists of a top-most dispatcher, responsible for packet formatting. Below the dispatcher, the protocol layer handles acknowledgments, error verification with CRCs, and windowing. The encoder/framer layer below the protocol layer (and over the raw UART layer) is responsible for translating raw bytes into frame and control information.

3.

DESIGN

Here we examine the design of the boot loader to better understand how SF C interacts with the various components and facilitates fast and reliable data transfer.

3.1

Boot Loader

The boot loader enables ISP-free reprogramming (but must be installed using an ISP initially). In MCUs which support self-programming [1], the program flash memory space (e.g., on-chip NAND Flash) is typically divided into two sections, the application program section and the boot program section, as shown in Figure 1. The boot loader is programmed into the boot program section using an ISP. By programming the BOOTRST fuse1 , the MCU can be configured to make it jump to the boot loader address at reset.

Figure 1: Program Flash Memory Layout2 The various components of the boot loader are shown in Figure 2. The boot loader receives the application program from the development system, using either a Wi-Fi module (over the air) or a RS232/UART module (over the wire). In this paper, we focus only on modules which terminate in a serial device. The serial communication layer is responsible for interacting with the Wi-Fi or the RS232/UART module. The application program is sent by a C program executing on the development system, running Linux, using either the TCP/IP socket library (Wi-Fi) or the Termios terminal I/O library (RS232/UART) for data transmission. The application data sent to the boot loader is in the 16-bit Intel HEX format [8], as shown in Figure 3. When a block of application data is received on the mote, the hex file parser is responsible for parsing/translating the received data. The central I/O module is responsible for storing the data in a temporary buffer of limited size. Since flash writes 1 BOOTRST is specific to the AVR family; other processor families also provide similar configuration features. 2 Reproduced from [1]

Figure 2: Boot Loader Components

Figure 3: Intel Hex File Layout

are performed a page at a time – typically 256B in embedded devices – the temporary buffer size is set to match it. When the temporary buffer becomes full, the central I/O module proceeds to write the data to on-chip flash. Alteratively, the data can be stored temporarily in the on-chip EEPROM, or an external NAND flash module. In this case, on completion of the data transfer, the on-chip flash can be programmed from the externally stored data. The central I/O module must block data flow after it has recorded enough data to fill its buffer, until it finishes writing the last page to flash. Again, after the write completes, the module must restart the paused data transmission. For these flow control requirements, the module uses SF C.

3.2 A Simple Flow Control Protocol The SF C protocol provides the ability for any program (boot loader in this case) to initiate and terminate data flow when required, while providing high baud rates and reliability against bit errors. Data transmission using SF C involves a connection setup phase, followed by a transfer phase, and finally, a connection teardown phase, as shown in Figure 4. In the event of an unsuccessful transfer, control moves back to the connection setup phase. SF C does not use the concept of frames; the granularity of data transfer is one byte. The right half of Figure 4 illustrates a typical sender–receiver interaction. SF C uses a

Figure 4: Operative Phases of SF C

simplistic, TCP-like, three-way handshake strategy during the connection setup and teardown phases. However, unlike TCP, where control messages consume an entire frame, SF C control messages (SYN, SYN-ACK, ACK, FIN, and FIN-ACK) are byte length ASCII codes. SF C uses a variation of the stop-and-wait flow control model for the actual data transfer, where the transfer granularity is set by the receiver. SF C does not allow control of the transmission data rate; the rate of data transfer is implicitly controlled by the receive buffer size. It allows the receiver to specify the amount of data that it is capable of receiving, and then initiates a transfer for the specified amount of data. Each such transfer is treated as an independent transfer. SF C ensures reliability for each transfer with the help of error-detecting codes (CRCs) and retransmissions. In the connection setup phase, the receiver initiates a connection by informing the sender when it is ready to accept data. Subsequently, the receiver and sender negotiate the number of bytes of data to be transferred. The receiver advertises the size of its buffer, while the sender advertises the size of the data it can send; the minimum of these two values is chosen as the data transfer size. In the data transfer phase, the sender sends the determined amount of data to the receiver. It also sends a CRC value, computed over the data block, to be used for data verification on the receiver end. On completion of successful data verification, both the receiver and sender enter the connection teardown phase. In the event the verification fails at the receiver, both the receiver and sender restart connection setup. The connection teardown phase consists of the receiver indicating to the sender that data verification was successful, the sender acknowledging the receipt of the message, and the receiver finally terminating the (logical) connection.

4.

IMPLEMENTATION

The SF C implementation consists of a receiver residing on the mote, and a sender residing on the development system. The code for the connection setup, data transfer, and teardown phases resides on both the receiver and sender. The SF C programming interface is shown in Listing 1. sfc init() is used to initialize the serial connection. The uart config t struct, contains the serial configuration data. This includes the baud rate, parity selection, number of data bits, and number of stop bits. sfc open() is responsible for

waking the UART from power reduction mode; the UART will be re-initialized based on the configuration data passed in the last call to sfc init(). sfc close() is responsible for putting the UART back into power reduction mode. 1 2 3 4 5 6 7

provides sfc send() the address and size of the source buffer (SND WIN). To setup the connection, the receiver starts at

void sfc_init ( uart_config_t * config ); void sfc_open (); void sfc_close (); uint8_t sfc_send ( uint8_t * buf , uint8_t size ); uint8_t sfc_receive ( uint8_t * buf , uint8_t size );

Listing 1: SF C Programming Interface

Figure 6: SF C (Send) State Machine

Figure 5: SF C (Receive) State Machine Figures 5 and 6 illustrate the receiver and sender state machines used by SF C, respectively. The states in the receiver machine are labeled R1 to R8 ; those in the sender state machine are labeled S1 to S10 . The numbers shown below the Send and Timed Receive annotations denote byte count, the number of bytes to be sent or received.

4.1 Connection Setup Phase The receiver initiates a connection by calling sfc receive(). A corresponding call to sfc send() is made on the sender’s side. The receiver provides sfc receive() the address and size of the target receive buffer (RCV WIN). The sender

R1 and sends SYN and RCV WIN back-to-back. It then waits for a given amount of time at R2 to receive SYN-ACK and SND WIN, failing which, it moves back to R1 . It then re-advertises SYN and RCV WIN. The sender starts at S1 and waits to receive the SYN and RCV WIN sequence. When received, it records the receive buffer size and moves to S2 , where it waits for another set of SYN and RCV WIN messages. It again records the receive buffer size received, and then compares the two RCV WIN values received in S3 ; on a mismatch, it moves back to S1 and begins again. On a match, the sender sends SYN-ACK and SND WIN in S4 and waits for a fixed amount of time in S5 to receive an ACK from the receiver. On the receiver side, on successful reception of SYN-ACK and SND WIN in state R2 , an ACK is sent back to the sender in R3 , indicating a successful connection negotiation.

4.2 Data Transfer Phase After the ACK transmission in R3 , the receiver enters a timed receive state for n + 1 bytes of data in state R4 , where n is the smaller of RCV WIN and SND WIN; one byte is added for a CRC. On receiving the ACK in S5 , the sender starts sending n bytes of data in state S6 , where n again stands for the smaller of RCV WIN and SND WIN. Once all the data has been sent, the sender transmits an 8-bit CRC computed over the data block just transferred. If n bytes of data are not received within the alloted time, the receiver moves back to the start of the connection setup phase in R1 . On receiving all n bytes of data, the receiver transitions to R5 , where it computes the CRC over the received data block and compares it to the CRC value received. On a mismatch, it moves back to the start of the connection setup phase in R1 . On a successful match, it moves on to R6 , the start of the connection teardown phase.

4.3 Connection Teardown Phase The connection teardown phase consists of the receiver sending a FIN in state R6 , if it has successfully received and verified the data from the sender. The receiver then moves on to the R7 state, where it waits for a limited amount of time for a FIN-ACK. The sender keeps waiting in S7 for a FIN. On receiving the FIN in S7 , the sender moves to the state S8 , where it sends a FIN-ACK. If the sender receives a SYN in state S7 , it goes back to S1 and proceeds to retransmit the current data block. If the receiver receives the FIN-ACK in R7 , it proceeds to the end of reception state in R8 and terminates the connection. If the receiver does not receive the FIN-ACK within a stipulated time, it resends the FIN from R6 and waits for the FIN-ACK in R7 again. After sending the FIN-ACK in S8 , the sender enters a timed receive state in S9 (a multiple of the timed receive period in R7 ). If the sender receives a FIN in S9 , it assumes the FIN-ACK was lost, and hence moves back to state S8 and resends the FIN-ACK. If the sender does not receive a FIN in a fixed amount of time in S9 , it assumes that the FIN-ACK transmission was successful and hence moves on to the end of transmission state in S10 .

5. EVALUATION Our design goal was to develop a simple flow control protocol that enables fast and reliable data transmission. To evaluate how SF C performs in view of these goals, we report on the results of several experiments.

5.1 Experimental Setup We implemented the boot loader and SF C protocol for the MoteStack, a state-of-the-art in-situ sensing platform, using C. We tested our implementation using the FTDI RS232/UART module [5] so that all TCP/IP induced time delays could be eliminated. The FTDI RS232/UART module was operated at variable baud rates from 9600 to 115200, driven by the Termios terminal I/O library for data transmission on a Linux development system. The serial configuration used was 8 data bits, no parity bits, and 1 stop bit (8-N-1). For every 8 data bits, 2 framing bits (1 start and 1 stop bit) are sent; hence the achievable data rate would be 80% of the baud rate. Baud rate errors introduced due to the MCU’s 10MHz clock rate must also be accounted for. The actual baud rates, data rates, and associated error percentages used in our experiments are shown in Table 1.

Table 1: Serial Data Transfer Characteristics Baud Rate (bits/second) 9600 19200 38400 57600 115200

Data Rate (bits/second) 7680 15360 30720 46080 92160

Error (%) -0.16 -1.74 -1.77 +1.52 -9.60

In each trial, we send a 32KB file containing random data from a development system to the mote. The file size reflects half the size of the flash program memory. We vary the receiver buffer size from 16B up to 256B, in combination with the different baud rates noted in Table 1. We conduct each experiment three times, measure the average time required to send the complete file, and calculate the effective data transmission rates achieved. All measurements are conducted on the sending side. We also consider protocol overhead and channel utilization. To ensure reliability, we compute a CRC over the entire file transferred and compare the calculated CRC with the CRC value received by the mote.

5.2

Performance

We now examine the throughput achieved using SF C. Figure 7(a) summarizes the achieved throughput for variable receiver buffer sizes. Figure 7(b) summarizes the impact of baud rate on throughput, for fixed buffer sizes. The figures show that the throughput increases with increasing buffer size and increasing baud rate. The peak data transfer rate achieved is around 15 Kbps, for a buffer size of 256B and a baud rate of 115200. The rate of change in observed throughput as a function of the buffer size in Figure 7(a) is almost logarithmic; it does not scale as expected from Table 1. The throughput change is pronounced for higher baud rates, since transmissions in the lower baud rates achieve lower peak values. Figure 7(b) shows that for smaller buffer sizes, increasing the baud rate does not cause a significant increase in throughput. Figures 7(c) and 7(d) summarize the channel utilization for various buffer sizes and baud rates, respectively. Channel utilization is calculated as the percentage of the achievable throughput (for baud rates shown in Table 1) that SF C was able to obtain. Channel utilization reaches a minimum at 115200 bps, while the throughput reaches a maximum. The highest network utilization is around 69%, when using a 256B buffer at 9600 bps. Since channel utilization increases with an increase in buffer size, the buffer sizes used for higher baud rates should be larger for attaining higher channel utilization. However, for system reprogramming purposes, we are limited to a maximum buffer size of 256B due to memory constraints. When a transmission succeeds without retransmissions, the protocol overhead introduced due to SF C is 8 bytes. This translates to 96.97% efficiency for a buffer size of 256B, and 66.67% efficiency for a buffer of 16B.

5.3 Reliability We ran a total of 75 experiments. During each trial, we recorded the number of transmission retries encountered due to bit errors. Surprisingly, there were no such retries. We posit that this is due to the high quality crystal used in our development system. At high speeds, on embedded devices that use inexpensive crystals (subject to humidity and

16000

9600 19200 38400 57600 115200

Throughput Obtained (bps)

14000 12000

bps bps bps bps bps

10000 8000 6000 4000 2000 0 0

50

100

150

200

250

300

100000

120000

Receive Buffer Size (B)

(a) Throughput vs Buffer Size 16000

16B 32B 64B 128B 256B

Throughput Obtained (bps)

14000 12000 10000 8000 6000 4000

SF C could provide end-to-end reliability and be used as the base for a network reprogramming strategy. By incorporating SF C within a persistent, epidemic data dissemination protocol (like Deluge [7]), and adding differential updates, we could design a fast, robust, and effective network reprogramming solution. In this paper, we presented SF C, a simple flow control protocol in the context of an embedded network system reprogramming strategy. We designed, implemented, and deployed a boot loader which utilizes SF C. We evaluated the SF C protocol and showed that it is capable of fast data transfer rates, while introducing minimal protocol overhead and providing reliability against bit-errors. Thus we have demonstrated a flow control mechanism which can form the basis of a reliable and scalable reprogramming strategy. Acknowledgments. This work was supported in part by the National Science Foundation (CNS-0745846).

2000 0 0

20000

40000

60000

80000

Baud Rate (bps)

(b) Throughput vs Baud Rate 70

9600 19200 38400 57600 115200

Channel Utilization (%)

60

bps bps bps bps bps

50 40 30 20 10 0 0

50

100

150

200

250

300

Receive Buffer Size (B)

(c) Channel Utilization vs Buffer Size 70

16B 32B 64B 128B 256B

Channel Utilization (%)

60 50 40 30 20 10 0 0

20000

40000

60000

80000

100000

120000

Baud Rate (bps)

(d) Channel Utilization vs Baud Rate Figure 7: Results temperature-induced variation), we expect the number to be higher. However, we did observe a number of synchronization retries since the sender and receiver were not synchronized. These were completely masked by our implementation. This is essentialy the same mechanism used to deal with bit errors, should they occur. Consequently, the reliability results appear favorable in our experimental trials.

6. CONCLUSION SF C can be further improved to support network reprogramming. The current SF C implementation is used between a development system and an embedded network system, because SF C can handle only bit-level errors. In scenarios where a full byte is lost, the current SF C state machine may not recover from the error. Under such circumstances, the only workaround is to restart the transmission. If this were remedied to support recovery after byte loss,

7.

REFERENCES

[1] AVR. Atmega datasheet. www.atmel.com/dyn /resources/prod documents/doc2593.pdf, 2010. [2] Rone Il´ıdio da Silva et al. Spatial query processing in wireless sensor network for disaster management. In Proceedings of the 2nd IFIP conference on Wireless days, WD’09, pages 194–198, Piscataway, NJ, USA, 2009. IEEE Press. [3] Philip Levis et al. Tinyos: An operating system for sensor networks. In Ambient Intelligence. Springer Verlag, 2004. [4] Philip Levis et al. T2: A second generation os for embedded sensor networks. Technical report, 2005. [5] FTDI. Ft232r datasheet. www.ftdichip.com/Support /Documents/DataSheets/ICs/DS FT232R.pdf, 2010. [6] EE Herald. Online course on embedded systems. http://www.eeherald.com/section/design−guide /esmod8.html, 2006. [7] Jonathan W. Hui and David Culler. The dynamic behavior of a data dissemination protocol for network programming at scale. In In Proceedings of the 2nd international, pages 81–94. ACM Press, 2004. [8] Intel. Intel hexadecimal object file format specification. www.microsym.com/editor/assets/intelhex.pdf, 1988. [9] M. Johnson et al. A comparative review of wireless sensor network mote technologies. In Sensors, 2009 IEEE, pages 1439 –1442, oct. 2009. [10] Sukun Kim et al. Wireless sensor networks for structural health monitoring. In Proceedings of the 4th international conference on Embedded networked sensor systems, SenSys ’06, pages 427–428, New York, NY, USA, 2006. ACM. [11] Telecom Corner Technical Reference Site. Data communications protocols. http://telecom.tbi.net /protocol.htm, 2004. [12] Geoff Werner-Allen et al. Fidelity and yield in a volcano monitoring sensor network. In Proceedings of the 7th symposium on Operating systems design and implementation, OSDI ’06, pages 381–396, Berkeley, CA, USA, 2006. USENIX Association. [13] James M. Westall. Class notes from cpsc 851. http://www.cs.clemson.edu/∼westall/851 /notes/link-layer.pdf, 2010.