It offers a user-friendly API that is very intuitive ... This API should also handle all platform dependent issues ..... [1] âJava(tm) 2 SDK, standard edition documenta-.
A multi-platform programming interface for protocol development Carlos Miguel Tavares Calafate and Pietro Manzoni Department of Computer Engineering Polytechnic University of Valencia (UPV) Valencia, SPAIN calafate, pmanzoni @disca.upv.es
Abstract We present a programming interface, called PICA, which aims to reduce the production cycle duration for communication protocols. It offers a user-friendly API that is very intuitive and that covers the major necessities which these protocols may have. Since most research is done in the Linux operating system, it also aims to reduce very significantly the time required to port a protocol implementation to other platforms like Windows NT or Windows CE by providing source code compatibility. We estimated the efficiency of our library and found that the overhead introduced is very small.
1 Introduction Reducing the development time and the testing cycle without affecting the overall performance is an highly desirable target when designing a new software product. From the conceptual definition of the product we would like to move, as fast as possible, to the implementation of a skeletal system, which is used to test the key features of the design. From this prototype and through a possibly long cyclic sequence of user evaluation and concept refinement and implementation of refined requirements we will eventually get to the final implementation. Rapid prototyping applied to network-based applications is also a very challenging goal. An ideal application programming interface (API) should be simple enough to be easily learned and used by programmers but it should also be as complete as possible to offer a full set of services. This API should also handle all platform dependent issues for the application, allowing it to be moved to different systems without requiring any special porting procedures. The already available solutions to achieve multi-platform compatibility are not entirely satisfactory because they do not provide an API that is useful to protocol designers, especially due to the restricted nature of functions that such a
designer needs. The Java technology [1], for example, not only lacks of such an API, but it also slows down the overall performance of the application due to strong encapsulation [2, 3]. The Cygwin [4] project offers another possible solution but it neither provides some specific kernel calls, neither does it provide new functions in order to aid in protocol development tasks. The large size of its accompanying DLL can also be a considered a drawback. We describe a protocol implementation specific library (PICA) which served us to ease the work of porting of the Ad hoc Protocol Evaluation (APE) testbed [5] to the Windows 2000 and Windows CE platforms. APE is a Linux based testbed for the evaluation of mobile ad hoc routing protocols which makes the process of performing complex real-world tests as easy as possible. This testbed includes the most common implementations of protocols for mobile connectivity, like Ad hoc On Demand Distance Vector [6], Dynamic Source Routing [7], Optimized Link State Routing [8], Lightweight Underlay Network Ad-hoc Routing [9] and Temporally Ordered Routing Algorithm [10]. These implementations were analyzed in order to find out their requirements. The Wireless Tools for Linux [11], an excellent work by Jean Tourrilhes, as well as his tutorial [12], served us to gain insights into the possibilities of a specific API for wireless connectivity. The work of Moustafa A. Youssef [13], which intends to provide a simple API to control wireless devices, but strictly under Linux, was also of great help. Though we had special interest in protocols for wireless network we maintained our proposal as generic as possible. We especially focused on the interoperability issues. The PICA library is based on the Packet Capture library (libpcap) work by Van Jacobson et al. [14] and its porting, Winpcap, to the Windows operating system by Fulvio Risso and Loris Degioanni [15]. Both works provide packet capturing and handling libraries, for distinct platforms, allowing to perform network analysis and to ease the implementation of communication protocols. From now on we will use the term pcap to refer to any of them.
Application
User App.
source code
PICA
PICA
PICA
PICA
Linux
Windows NT
Windows CE
Other OS
Figure 1. Source code compatibility through PICA
PCAP library
PICA
Kernel Figure 2. Designer API choices
The PICA library was created to provide a multiplatform API to communication protocols designers. The objective was to accelerate the prototyping phase to provide programmers with stable solutions in a shorter time, as well as designing solutions whose source code compiles directly on distinct platforms, as illustrated in figure 1. The PICA architecture tries to be efficient in terms of code size and speed, making the differences in performance when compared to a customized solution minimal. The PICA API was developed in ANSI C language and is available as a dynamic-link library (.dll) file for Windows operating systems and as a shared object (.so) file for Linux based operating systems. PICA is released under the GNU General Public License (GPL). The rest of this paper is organized as follows. Section 2 presents a general description of the library structure. Section 3 describes a detailed analysis of the performance of the library in various different typical situations.
2 Overall architecture The PICA library creates an adaptation layer between the user and the kernel space and offers specialized functions that aid the programmer activity when creating networking solutions based on packets capturing and handling. PICA, as illustrated in figure 2, does not hide completely the underling layers. Depending on the needs of the protocol being designed, we could use the PICA interface, the pcap library or directly the kernel. The use of the pcap libraries does not compromise inter-platform compatibility, since as described in the previous section, there are pcap versions available for all the platforms where PICA is available. By using kernel specific functions, though, the source code will loose its compatibility between platforms, and porting it will require and extra effort on the developer side. Making PICA available for distinct platforms is an attractive goal, although we had to impose a few restrictions due to the sometimes radically different approaches taken by the OS architects. The most relevant restrictions will be
commented where they apply. The PICA architecture is divided into two sets of primitives, namely the system management primitives and the networking management primitives. The former deals with the different approaches an operating system takes to handle its resources, while the latter deals with data packets and routing structures. Appendix A presents a complete list of the functions that belong to each set of primitives.
2.1 The system management primitives Two basic tasks are quite often used when designing protocol implementations: process management and memory management. The fork() call is of common use in Unix environments to manage processes. Windows systems, though, do not offer this function. We therefore adopted a combination of the threads approximation with the semaphore and mutex abstractions as an alternative to processes without generating too much extra code. Although the Posix standard [16] does not allow thread suspension and resuming, the PICA library allows the use of such functions in the Linux operating system by means of the SIGUSR1 and SIGUSR2 signals. This solution tries to cope with the differences with respect to the Windows kernel where such functions exist. The recommended practice is anyway to avoid such calls because they can produce unpredictable results in critical sections of code. Also, the Posix standard implementation does not allow setting the maximum value of a semaphore, which is a feature available in Windows. PICA also provides such functionality, though introducing a little overhead. The select function is a well known Unix system call used to wait for events associated with any kind of descriptor; descriptors are represented by integers values. Instead, in Windows operating systems, the descriptors are generally represented by a specific data type called HANDLE, while integers are only used for sockets. In Windows the select function is only available for sockets, while for others
Call to PICAselect()
PICA internals Creation
WaitForMultipleObjects()
Event
on HANDLES
Pipe
select() on Sockets
PICAselect() result
of unifying the different data types, i.e., HANDLE in Windows and int in Linux, used as file descriptor. Finally, PICA has a very reduced set of administrative functions used for such tasks as to initialize and release the library, to get a list of the devices available, to activate Windows Sockets API, or to test if the current user has certain privileges. PICA also provides a straightforward way to handle errors. Since all functions return either or in case of error or of success respectively, every time an error occurs the PICA library is actualized and a call to a specific function (PICAgetLastError()) returns a string containing the error message, as well as the error number. We chose this technique since it is practical and widely used.
2.2 Networking management primitives Figure 3. Architecture of the PICAselect() function in Windows
events we have to use a function of the WaitFor family. The PICA library obviates this problem by emulating the Linux behaviour. When invoking the PICAselect() function the current thread stops waiting for events using a WaitForMultipleObjects() call and a secondary thread is created. This thread is dedicated uniquely to socket handling. When an event occurs on a socket, the secondary thread informs the waiting thread, and through a system pipe it sends the information about which socket is active (see Figure 3). Relatively to memory management, the PICA architecture is based on offering the possibility to easily handle a queue data structure. The objective was to provide auxiliary functions which could be useful at implementation time. The programmer can create as many queues as desired, allowing differentiated packet handling. The architecture chosen allows the use of multiple groups of queues, each group having a number of queues chosen by the user. In order to guarantee data coherence to multi-threaded applications, PICA uses a different mutex status variable to control the access to each queue so that, for example, distinct threads can read and write to different queues, even though these queues belong to the same group. It is relevant to point out that in order not to generate meaningful delays, these routines do not perform any kind of buffer duplication, having the sole task of managing pointers to the data. PICA offers a generic pipe data structure, too. It is a good practice to include logging capabilities in the applications that are developed, especially when these applications are network daemons. The PICA logging functions offer straightforward methods to read from and write into files. The new file type created has the unique purpose
The basic data structure abstraction is the data packet. PICA offers a small set of functions to send and receive (capture) packets. The approach taken in Linux and Windows are quite different. In Windows we inherited the functions to send and receive packets offered by the PACKET.DLL library which is part of the winpcap distribution. While Linux does not offer such a library it provides direct read and write access to a network interface (NIC) by using the PF PACKET socket domain. The socket interface is also one of the most frequently used interfaces when developing protocols implementations. PICA encapsulates this API using macros and eliminates the apparently slight differences between the various platforms. It should be noticed, though, that the Windows Sockets Version 2 (WinSock 2) API [17] does not provide useful functions such as the recvmesg and sendmsg whereas in Linux they are available. WinSock 2 also lacks of some of the options that are available on Linux based implementations. These factors should be taken into account by programmers, because they can compromise the compatibility between platforms. Handling timers is also of extreme relevance. Protocols and applications are often required to perform scheduled actions, e.g., to take into account network congestion status. Differently from Windows, the Linux architecture imposes a single timer per application. This ”restriction” encouraged us to provide PICA with multiple timers, by means of a priority queue. In this queue all events make use of the same timer in a way that only the first to-happen event affected will be used to set the value of the timer. There is a thread that executes the code indicated by the user, allowing the event handling to be not only asynchronous, but also parallel. The solution provided allows high versatility due to the possibility of multiple call-back functions, each accepting an argument. This allows for a differentiated approach to events, enabling the designer to group events into families, achieving more elegant solutions to common problems.
Finally, PICA provides functions to handle the IP routing functionalities. The PICA routing functions allow the designer to add and remove entries in the forwarding table. These functions are not frequently used, but we found that they are sometimes useful in protocol design to provide dynamic connectivity. See, for example, the AODV implementation [6]. Moreover some protocols need to check some of the entries, like the broadcast ones, and this can be easily accomplished using these functions. We also give the possibility to get access to the current forwarding status and to the time-to-live (TTL) attributes. It should be noticed that the TTL value affected is the global system’s TTL. Lowering this value too much might cause loss of connectivity to other networks (e.g., Internet). For a per-socket definition of this value, the socket options available in most systems should be used.
3 Experimental Results In this Section we evaluate the efficiency of PICA. Various test sequences were run on a standard PC with both Mandrake Linux 8.0 and Windows 2000 Professional installed. The machine has a 750 Mhz AMD Athlon(tm) Processor with 256 MB of RAM. The results obtained were achieved by executing each test sequence one million times, comparing the results using either the system native functions and the functions provided by PICA, and getting the time (in ms) to completion of a particular sequence. The analysis concerned the latency generated by some frequently used functions. The chosen test operations were: a file reading test, a file writing test, a socket test, a pipe test and finally a thread test. The file reading test consisted of opening, reading a byte and closing the file; the file writing test consisted of simply writing a byte to a file and the socket test consisted of opening and closing a socket. Concerning the pipe test, it consists of creating a pipe, writing a byte and reading it in the other end, and finally closing both ends. In the thread test a single thread is created and killed. Since the library also provides synchronization primitives, other set of test operations, which consisted of: suspending and resuming a thread, acquiring and releasing a mutex as well as acquiring and releasing a semaphore, was also included. The results of these tests are displayed in figures 4 and 5, and the overhead values are displayed in tables 1 and 2. As can be seen, the difference of performance between the use of native functions or PICA is quite small, being almost unnoticed when compared to the differences between O.S.; it should be noticed, though, that the comparison of performance between O.S. is not our goal, being just a side effect of the current analysis. The most relevant difference between native calls and
Table 1. PICA overhead on Linux O.S. Operation % increase Open, read & close 4,45% Write to file 2,62% Socket open / close 0,6% Pipe operations 3,9% Thread create / kill 0,06% Thread suspend / resume N/A Mutex acq. / release 37,4% Semaphore acq. / release 87,33% Table 2. PICA overhead on Windows O.S. Operation % increase Open, read & close 0,34% Write to file 7,8% Socket open / close 1,8% Pipe operations 1,74% Thread create / kill 0,16% Thread suspend / resume 10,84% Mutex acq. / release 10,16% Semaphore acq. / release 12,97% PICA is shown by the semaphore calls for the Linux O.S.; such difference is not due to code inefficiency, but to the enhancement introduced to allow the programmers to specify a maximum count value for a semaphore, as happens in Windows.
4 Conclusions We described a protocol implementation specific library (PICA) which provides a multi-platform API to communication protocols designers. The objective was to accelerate the prototyping phase to provide programmers with stable solutions in a shorter time, as well as designing solutions whose source code compiles directly on distinct platforms. We showed that the PICA architecture is efficient in terms of code size and speed, making the differences in performance when compared to a customized solution minimal. The PICA API was developed in ANSI C language and is available as a dynamic-link library (.dll) file for Windows operating systems and as a shared object (.so) file for Linux based operating systems. PICA is released under the GNU General Public License (GPL). Future developments may include the extension of this library to other Unix environments, as well as implementing some of the well known protocols for mobile connectivity using PICA. The version of this library for Linux, Windows and Windows CE platforms including the library documentation and other useful links are available at: http://picaapi.sourceforge.net/.
References [1] “Java(tm) 2 SDK, standard edition documentation,” Sun Microsystems, Inc, 2002, Available at http://java.sun.com/j2se/. [2] Ambika Pajjuri and Haseeb Ahmed, “A performance analysis of java and c,” University of Columbia, 2001, Available at http://www.cs.columbia.edu/˜sedwards/. [3] Milo Martin, Manoj Plakal, and Venkatesh Iyengar, “Top-level data-memory hierarchy performance: Java vs. c/c++,” University of Wisconsin Madison, Dec 1996, Available at http://citeseer.nj.nec.com/210886.html. [4] Geoffrey J. Noer, “Cygwin: A free win32 porting layer for UNIX Applications,” August 1998, Cygwin distribution is available at http://cygwin.com/. Figure 4. Performance chart for Linux and Windows 2000 implementations
[5] Henrik Lundgren, David Lundberg, Erik Nordstr¨om, Christian Tschudin, and Johan Nielsen, “A largescale testbed for reproducible ad hoc protocol evaluations,” Proceedings of the IEEE Wireless Communications and Networking Conference (WCNC 2002), March 2002. [6] Charles E. Perkins, Elizabeth M. Belding-Royer, and Samir R. Das, “Ad hoc on-demand distance vector (AODV) routing,” Internet Draft, MANET Working Group, draft-ietf-manet-aodv-10.txt, January 2002, Work in progress. [7] David B. Johnson, David A. Maltz, Yih-Chun Hu, and Jorjeta G. Jetcheva, “The dynamic source routing protocol for mobile ad hoc networks,” Internet Draft, MANET Working Group, draft-ietf-manet-dsr-07.txt, February 2002, Work in progress. [8] Thomas Clausen, Philippe Jacquet, Anis Laouiti, Pascale Minet, Paul Muhlethaler, Amir Qayyum, and Laurent Viennot, “Optimized link state routing protocol,” Internet Draft, MANET Working Group, draftietf-manet-olsr-06.txt, September 2001, Work in progress.
Figure 5. Performance chart II for Linux and Windows 2000 implementations
[9] Christian Tschudin and Richard Gold, “Lightweight underlay network ad-hoc routing,” January 2002, Available at http://www.docs.uu.se/. [10] V. Park and S. Corson, “Temporally-ordered routing algorithm (TORA) version 1 - functional specification,” Internet Draft, MANET Working Group, draftietf-manet-tora-spec-03.txt, November 2000, Work in progress.
[11] Jean Tourrilhes, “Wireless extensions for linux,” Hewlett Packard Laboratories, Palo Alto, 1996, Available at the author’s home page at http://www.hpl.hp.com/.
A Appendix A.1 Device int PICAgetAvailableDevices(DEVLIST * devs);
[12] Jean Tourrilhes, “Linux wireless LAN howto,” Hewlett Packard Laboratories, Palo Alto, August 2000, Available at the author’s home page at http://www.hpl.hp.com/.
int PICAgetDeviceAttrs(char * dev, DevAttrs * attrs);
A.2 Route
[13] Moustafa Youssef, “MAPI: An API for wireless cards under linux,” Department of Computer Science, University of Maryland, Available at http://www.cs.umd.edu/˜moustafa/. [14] V. Jacobson, C. Leres, and S. McCanne, “The libpcap packet capture library,” Lawrence Berkeley Laboratory, Berkeley, CA, Available at http://www.tcpdump.org.
int PICAaddRoute(UINT32 dst, UINT32 mask, UINT32 gateway, int metric, char * device);
int PICAdelRoute(UINT32 dest, UINT32 mask, UINT32 gateway, char * device);
int PICAgetRoutingTable(struct RTInfo * rti);
A.3 IP Forwarding
[15] Fulvio Risso and Loris Degioanni, “An architecture for high performance network analysis,” Proceedings of the 6th IEEE Symposium on Computers and Communications, pg. 686-693, Hammamet, Tunisia, July 2001. [16] IEEE, “Portable operating system interface (posix) - part 1: System application programming interface (api) [c language],” ISO/IEC 9945-1, 1996.
int PICAisForwarding(int * true false);
int PICAsetForwarding(int on off);
int PICAdefaultTTL(int set get, int * ttl);
A.4 Packet int PICAopenDevice(char PICA IO DEVICE * iodev);
[17] Martin Hall and Dave Treadwel et al., “Windows sockets 2 application programming interface,” August 1997, Available at ftp://ftp.microsoft.com/.
*
device,
int PICAframe(int mode, PICA IO DEVICE iodev, void * packet, int packetsize, int * read);
int PICAcloseDevice(PICA IO DEVICE iodev);
A.5 Socket int PICAcreateSocket(PICAsocket * sd, int domain, int type, int protocol);
int PICAcloseSocket(PICAsocket sd);
A.6 Threads int PICAstartThread(THRID * thr, void * func, void * arg);
int PICAsuspendThread(THRID thr);
int PICAresumeThread(THRID thr);
int PICAkillThread(THRID thr);
int PICAcreateMutex(PICAmutex * mut);
int PICAmutexAction(int action, PICAmutex mut);
A.10 User info
int PICAdestroyMutex(PICAmutex mut);
int PICAcreateSemaphore(PICAsemaphore * sem, int initial count, int max count);
int PICAsemaphore(int action, PICAsemaphore sem, int count);
int PICAisAdministrator(int * user type);
A.11 Library specific int PICAstartup(int flags);
int PICAdestroySemaphore(PICAsemaphore sem);
int PICAcleanup(void);
int PICAaddDesc(PICAdescList * dl, int type, int mode, void * desc);
int PICAselect(int time, PICAdescList * dl, PICAselResult * res);
A.7 Memory Management int PICAmakePipe(PICApipe * in, PICApipe * out);
int PICAsendToPipe(PICApipe out, void * data, int size, int * written);
int PICAgetFromPipe(PICApipe in, void * buf, int bufsize, int * datasize);
int PICAclosePipe(PICApipe pipe);
int PICAinitBuffer(PICAbuffer num queues);
**
buf,
int
int PICAaddToBuffer(PICAbuffer * buf, int queue id, void * data, int data size);
int PICAgetFromBuffer(PICAbuffer * buf, int queue id, int num packets, PICApacket ** packets, int * avail packets);
int PICAkillBuffer(PICAbuffer * buf);
A.8 Timer int PICATimer(int action, UINT64 * time, void * function, void * data);
INT64 PICAgetCurrTime(void);
A.9 Log int PICAopenFile(FDesc * fd, char * name, int read write, int flags);
int PICAreadFile(void * buf, int numbytes);
int PICAwriteToFile(FDesc file, void * data, int datasize);
int PICAcloseFile(FDesc file);
int PICAgetTimeStr(char ** time str);
void PICAgetLastError(char * message);
