Slotted-FIFO Channels for Asynchronous Distributed Systems Ravi Prakash Roberto Baldoni, Roberto Beraldi Department of Computer Science Dipartimento di Informatica e Sistemistica University of Rochester Universita di Roma \La Sapienza" Rochester, NY 14627-0226 Via Salaria 113, 00198 Roma, Italy.
[email protected]
fbaldoni,
[email protected]
Abstract Communication protocols designed for database applications are not necessarily suitable for other applications, like multimedia communication, due to the former's requirement of reliable and ordered communication, and the latter's ability to withstand occasional losses and misordering of messages as long as real-time communication can be supported. This paper presents the slotted-FIFO communication protocol that supports communication primitives for the entire spectrum of reliability and ordering requirements of distributed applications: FIFO as well as non-FIFO, and reliable as well as unreliable communication. Hence, the slotted-FIFO protocol is suitable for multimedia applications like MPEG video transmission and video telephony, as well as non real-time distributed applications. The protocol is simple and has low overheads. As FIFO ordering is not required for all messages, message bu ering requirements are considerably reduced. Also, message latencies are lower.
Index Terms: message ordering, reliable communication, FIFO-transmission, communication slots.
1 Introduction A major focus of recent research in data communication over computer networks has been to ensure a reliable and, if possible, ordered ow of data. The de nition of communication modes, such us FIFO [10] and causal ordering [5, 6], and ow control using windowing and positive/negative acknowledgments have been motivated by such research. At the conceptual level, communication subsystems have been usually modeled as being reliable with unpredictable, yet nite, message transmission time. Such a communication model is relevant for database applications that rely on ordered and reliable ow of data. Until recently, database applications were dominant among all applications executing in a networked environment. However, many applications require communication with relaxed reliability and/or ordering constraints. For example, multimedia communication over computer networks which has become a reality in the last few years, and is gaining in popularity. Multimedia applications have to handle audio and video streams of data that are played back in real-time at the destination site. These streams can tolerate some loss of information as long as these losses do not cause a degradation in the desired quality of service (QoS) for the application [9, 12]. Therefore, the ordering and reliability constraints imposed by database applications can be relaxed for multimedia applications. So, communication paradigms suitable for database applications may not necessarily be suitable for multimedia applications. New communication paradigms need to be de ned. One such communication paradigm is -causal ordering [3, 4, 11] that, assuming application messages have a lifetime, delivers as many messages as possible in their lifetime in such a way that these deliveries are causally ordered. A good example of the requirements of multimedia communication applications is the MPEG video compression standard [8]. MPEG compressed video consists of three kinds of frames. The Intraframe frames have no dependencies and can be decoded independently of other frames. A Predicted frame can be decoded only if the previous frame is available. A Bidirectional frame requires the closest Intraframe or Predicted frame before and after it for decoding. Thus, MPEG induces dependencies between some frames. It is obvious that the loss of some frames in a sequence may lead to an inability to decode all the subsequent 1
frames which depend on it, while the loss of another frame may not a ect the decoding of any frame. This leads to communication with di erent priorities, ordering and reliability constraints for di erent data units within the same data stream. A trivial approach to handle such a data ow could be to send data on two separate channels. The rst frames of distinct sequences are sent along a channel supporting reliable and FIFO delivery. Succeeding frames of the sequence (that are dependent on the rst frame) can be sent along a channel that can lose and/or reorder them. This solution raises issues pertaining to re-synchronization and maintaining dependencies among frames at the destination site of the two distinct data ows along di erent channels. In this paper, we introduce a new communication mode called slotted-FIFO ordering and give an implementation based on sequence numbers. This mode allows messages with di erent ordering and reliability constraints to be interleaved on the same channel while maintaining certain order dependencies among messages. In particular, we propose a set {send, F R {send, of synchronization primitives for sending messages, namely, FR{send, FR and F R {send, to be read as FIFO and Reliable send, non-FIFO and Reliable send, FIFO and non-Reliable send, and non-FIFO and non-Reliable send, respectively. For a message m, sent by process p to process q along the channel cp;q using an FR ? send primitive, the following properties are ensured at process q: 1) a message sent before m along cp;q , if delivered to q, is delivered before m, 2) a message sent after m along cp;q , if delivered to q, is delivered after m.1 If the message is sent using a reliable primitive, the delivery eventually occurs. Two successive message send events, corresponding to messages m and m , using the FR ? send primitive de ne a slot S along the channel cp;q . A message m1 sent using the ? send primitive, executed inside S, ensures that m1 is delivered after m and before m . FR A pair of message m1 and m2 sent using F R ? send primitives within S ensure that m1 and m2 , if delivered to q, are delivered after m, before m , and in their sending order. A message 1 The case where all messages are sent using the reliable primitive F R ? send corresponds to the two 0
0
0
way ush primitive proposed by Ahuja [1, 2] to increase concurrency on reliable channel compared to FIFO
channels.
2
m1 sent using the F R ? send primitive ensures that m1 , if delivered to q, is delivered after m and before m . Multiple messages sent during a slot using the F R ? send primitive can be delivered during the same slot in any order. The properties of the various send primitives are illustrated in Figure 1 where FR:i and FR:i +1 correspond to messages m and m mentioned above. 0
0
__ FR
FR
messages
FR
messages
FR.i+1
FR.i+1 FR.i
FR.i
messages
FR.i+1 FR.i
(b)
(a)
(c)
Figure 1: Semantics of slotted-FIFO send primitives. This paper is organized as follows: Section 2 introduces the system model and the communication primitives. Section 3 presents the slotted-FIFO communication mode and Section 4 shows an implementation based on sequence numbers. Two examples illustrating the behavior of the slotted-FIFO mode are presented in Section 5. Section 6 contains a discussion of some applications for which slotted-FIFO channels may be useful. The paper concludes with Section 7.
2 A Model of Distributed Executions A distributed computation is a nite set, P , of sequential processes that communicate and synchronize only by exchanging messages. Each process runs on a processor (for simplicity, we assume one process per processor). The processors have local clocks, they do not share memory. Failure handling is not considered in this paper. We assume that each ordered pair of processes p; q 2 P , is connected by a directed and asynchronous logical channel cp;q (p is the sender and q is the destination process). 3
Along this channel, a nite sequence of messages M are sent. The messages in the sequence can be labeled as m:0; m:1; : : : m:k, where m:i denotes the (i + 1)st message sent by p. This channel supports four primitives that an application can use to send a message in an {send, F R {send and F R{send. A ordered and/or reliable way: FR{send, F R{send, FR channel can drop messages. However, the underlying communication system is endowed with mechanisms such as positive/negative acknowledgment and retransmissions to ensure, if requested, reliable delivery of messages. The two reliable communication primitives can invoke these mechanisms. A message sequence sent along a channel can be divided into subsequences based on the ordering and reliability characteristics of the constituent messages. Let XY be a label be FR; F R g. Let XY:j be the (j +1)st message of the subsequence longing to the set fFR; F R; MXY sent using the XY ? send(m) primitive. The messages in this subsequence can be represented as XY:0; XY:1; : : : XY:kXY , where kXY k. Let there be a function f : MXY ! M which given the identity of a message sent by an XY ? send primitive determines the identity of the same message in M . We nally assume that m:0 = f (FR:0) and m:k = f (FR:kFR), i.e., the rst and last messages in the message sequence are sent using the FR ? send primitive. Now, we can de ne a slot as follows.
De nition 1 A slot Si is the set MSi of messages m:x (where MSi M ), sent by p along cp;q , such that f (FR:i) < m:x < f (FR:(i + 1)). where m:a < m:b implies that message m:a was sent before message m:b. Therefore, all messages sent by process p, along a channel cp;q , between two successive FR messages belong to a slot. The delivery of these messages produces a sequence of events d(m:x0 ); : : : d(m:xh) (with h k) on process q. We denote m:xh ; m:xw i d(m:xh) precedes d(m:xw ) in the sequence. Moreover, let (Si) be the set of messages m:x such that f (FR:i) ; m:x ; f (FR:(i + 1)), i.e., messages belonging to slot i that are delivered to process q. 4
3 The slotted-FIFO communication mode De nition 2 A pair of processes p; q respects slotted-FIFO ordering on cp;q i : 1. 8FR:h; FR:k 2 M :: (h < k) iff FR:h ; FR:k;
2 Si :: FR:k 2 (Si); 2. 8FR:k 2 (Si) :: F R:k 2 Si ; 3. 8F R:k F R:k 2 (Si) :: F R:h ; F R:k iff h < k; 4. 8F R:h; Examples of slotted-FIFO communications are depicted in Figure 2. Messages m0 and m5 , sent using the FR{send primitive, mark the beginning and end of one slot. Similarly, messages m5 and m10 mark the beginning and end of another slot. As messages m1 to m4 belong to the rst slot, if they are delivered their delivery should always follow the delivery of m1 and precede the delivery of m5 . As m3 and m4 are sent using the F R {send primitive, m4's delivery cannot precede m3 's delivery. As m1 and m2 do not have any ordering constraints placed on them, there are no delivery constraints among message sets fm1 g, fm2 g, and fm3 , m4 g. Loss of messages m3 and m6 in transit is acceptable as they are sent using nonreliable send primitives. p
m0
m1
m2
m3 m4
m5 m6 m7
m8
m9
m 10 m1
m0
m6
m3
m4
m2
q FR
FR
FR
-FR
m5
m8
m9
m7
Figure 2: An example of slotted-FIFO communication and the corresponding partial order generated by the relation 5
m 10
4 A Simple Implementation 4.1
The Sending Process
The implementation of the slotted-FIFO discipline adds to the original underlying system speci c controls. Messages are not necessarily delivered in the same order that they are received from the underlying network. Delivery of some messages may be delayed until some conditions are satis ed. This is done to ensure that message deliveries respect the slotted-FIFO communication mode.
Local information The sending process maintains the following three variables:
slot: the current slot number SFR : the sequence number of the next Non-FIFO and reliable message in the current slot
SF R : the sequence number of the next FIFO and unreliable message in the current slot
Information carried by each message Each message is equipped with control information stored in its ST structure. The elds of ST are as follows:
slot: integer indicating the slot associated with the message; type: a char indicating the type of the message; order: integer whose value depends on the type of the message; Implementation of the send primitive is given in Figure 3. First, elds slot and type of ST are updated (S1). When a reliable FIFO message has to be sent, the order eld is set to the current value of the SFR counter (S2). This is needed to inform the receiving process of 6
init SFR =0; =0;SF R procedure SEND(m; t; j ) slot=0;
/*m is the message, t is the type, j is the destination process*/
begin ST:slot = slot; ST:type = t;
case
t
(S1)
of
FR : ST:order = SFR = 0; ; SFR slot = slot + 1; SF R = 0; break; : SFR FR + 1; break ; = SFR F R : SF R = SF R + 1; ST:order = SF R ; break; F R : break;
(S2) (S3) (S4) (S5) (S6) (S7) (S8)
endcase
m ST );
send( ,
end.
(S9)
Figure 3: A simple send primitive implementation the number of reliable messages sent during the current slot (i.e., the slot being closed by the FR message). The FR message should not be delivered to the receiving process until all the preceding reliable messages in the slot have been delivered. The slot counter is incremented (S3), and the FIFO unreliable counter is reset (S4). When a non-FIFO reliable message has to be sent, the SFR counter is incremented. When sending an unreliable FIFO message, the SF R counter is incremented, and the order eld of ST is set to the current value of the SF R counter. 7
It is to be noted that a process need not maintain sequence number information for nonFIFO unreliable messages in the form of an SF R variable. Also, the order eld of the ST structure sent with such messages is unassigned. This is because the only constraint relevant for the delivery of F R class of messages is the slot number. The structure ST allows us to model messages as a partial order. This will be useful to de ne an ordering in the activation of multiple suspended WAIT conditions as explained later. 1. Let STm be the control information structure for the message m. Let m and m be two messages, we say that m precedes m , denoted m s m , i STm :slot < STm :slot. 0
0
0
0
2. Let m and m be two messages such that STm:slot = STm :slot and STm:type = STm :type = F R, we say that m precedes m , denoted m f m , i STm :order < STm :order. 0
0
0
0
0
0
3. Let m and m be two messages, we say that m precedes m , denoted m es m , i STm:slot = STm :slot and STm :type = FR. 0
0
0
0
0
Now, we denote as the transitive closure of the union of s, f , and es. The set of messages can then be represented as a partial order of messages M^ = (M; ). Two messages m and m are concurrent i :(m m ) and :(m m). In Figure 2.b the partial order M^ = (M; ) of the message scheduling of Figure 2.a is shown. 0
4.2
0
0
The Receiving Process
The receiving process manages the following four variables:
slot: an integer that stores the current slot number; RFR : an integer that stores the sequence number of the last reliable non-FIFO message received in the current slot; 8
RF R : an integer that stores the sequence number of the last unreliable FIFO message received in the current slot;
end slot: a Boolean variable indicating that an FR message has arrived and the end of the slot is imminent; Each time a message is received, an instance of a RECEIVE process is invoked at the receiver node to handle the message. The RECEIVE process executes the statements shown in Figure 4. Let the type of the message received be FIFO and reliable. Statement (R2) permits further processing of the message, and ultimately its delivery, only when the value of the local slot variable is equal to the value of the message's ST:slot eld and the message's ST:order eld is equal to the number of reliable messages sent in the current slot. This ensures that the reliable messages sent by the source in the current slot are delivered to the destination before messages belonging to the next slot are delivered. Both RFR and RF R counters are also reset (R3,R4). If a non-FIFO reliable message is received, the receiving process waits until the ST:slot eld equals the current slot (R8). The RFR eld is incremented to re ect the delivery of a non-FIFO reliable message (R9), and the message is delivered (R10). If an unreliable non-FIFO message is received and the value of its ST:slot eld is lower than the current slot, the message is discarded (R11, R12). Such a situation arises if the message has been overtaken on its way to the destination by a reliable FIFO message and arrives after the expiration of its slot. Otherwise it is delivered as soon as the the slot eld is equal to the current slot (R13,R14). Similarly, an unreliable FIFO message is discarded if the value of its ST:slot eld is either less than the current slot or if an FR message has arrived and the end of its slot is imminent (R15, R16). Otherwise its delivery is delayed until one of the the conditions in (R17) is satis ed. The rst condition (i.e., ST:slot == slot and ST:order==RF R + 1)) is the typical FIFO condition. The second condition (i.e., end slot) indicates that the end of the slot is 9
imminent. So, all received F R messages must be immediately delivered in the correct order before the received FR message be delivered.
Deadlock Avoidance If the WAITs in the case statements are busy-waits, deadlocks can arise. For example, let message followed by an FR message to the receiver. Also, let the FR a sender send an FR message. If busy-waits are employed at the receiver, the receiver message overtake the FR process will be spinning on the condition ST:order == RFR . On the arrival of the FR as the processor message the processor will not be able to handle it and increment RFR cycles are being monopolized by the busy-wait. Hence, it is important to ensure the following:
When an instance of the RECEIVE procedure reaches a WAIT statement and the condition in the statement is not true, that particular instance of the RECEIVE procedure is suspended and gives up control of the receiver process.
A suspended instance of the RECEIVE procedure is activated when the condition on which the procedure is waiting becomes true due to execution of statements in other instance(s) of the RECEIVE procedure at the same site.
If delivery conditions of multiple suspended instances of the receive procedure become true simultaneously (for example the ones associated with messages m and m ), they are activated in an order consistent with the partial order given by the relation , i.e., m is activated before m if m m . If m and m are concurrent they can be activated in any order. 0
0
0
0
Finally, to ensure consistency of updates to the condition variables the following statement sequences are executed in an atomic fashion: (R3) ? (R7), (R9) ? (R10), and (R18) ? (R19). We would like to remark that a busy-wait implementation using preemptable threads is possible. However, in that case we need to take care of priority among threads. 10
init RFR =0; RF R =0; end slot=false; procedure RECEIVE(m; ST ) slot=0;
/*m is the message, ST is the control information*/ begin
case FR :
ST.type
of
true;
end slot =
WAIT (slot==ST.slot
(R1)
and
RF R );
ST.order==
RFR =0; RF R =0;
(R3) (R4)
false;
: FR
(R2)
end slot=
(R5)
slot=slot+1;
(R6)
m
deliver( ); break;
(R7)
WAIT(slot==ST.slot);
(R8)
RFR + 1; =RFR deliver(m); break;
(R10)
(ST.slot < slot)
(R11)
F R : if
(R9)
then discard(m); break; else WAIT(ST.slot==slot);
(R12) (R13)
m
deliver( ); break;
F R : if
((ST.slot < slot)
or
(R14) end slot)
then discard(m); else WAIT((ST.slot==slot and
(R15) (R16)
RF R +1) or
ST.order==
RF R =ST:order; deliver(m);
end slot); (R17) (R18) (R19)
break;
endcase end. Figure 4: A simple receive primitive implementation 11
p
m0
m1
m2
m3
m
4
m5
q
FR
FR
FR
Figure 5: An example illustrating behavior of slotted-FIFO mode
5 Examples The behavior of the slotted-FIFO mode can be illustrated by the following examples. In the rst example, shown in Figure 5, process p sends six messages to process q. Messages m0 and m5 are of type FR, m1, m2 and m3 are of type F R , while m4 is of type . The messages are received by process q in the order shown in the gure. FR When m2 and m3 arrive at process q they cannot be immediately delivered as the values of their ST:order elds are greater than the expected value. This happens because m2 and m3 have overtaken m1. So, m2 and m3 are bu ered and the corresponding receive processes wait on the condition in line (R17). However, on the arrival of m5 end slot is set to true and both m2 and m3 become eligible for delivery. So, m2 is delivered before m3 as m2 f m3 . Message m5 cannot be delivered immediately as its ST:order > RFR due to the non-arrival of message m4 , and the delivery condition in line (R3) is not satis ed. When m1 arrives it nds that end slot is true. So, m1 is discarded. Had m1 been delivered after m2 and m3 the FIFO property would have been violated. When m4 arrives it is delivered as its delivery condition is satis ed. On the delivery of m4 the value of RFR is incremented. As a result the delivery condition in line (R3) becomes true and message m5 is delivered. Thus, slotted-FIFO mode ensures that reliable FIFO messages are not discarded and 12
p
m1
m2
m3
m4
q FR
FR
Figure 6: Behavior of slotted-FIFO mode on message loss. are delivered in the FIFO order. Unreliable FIFO messages may be discarded. However, unreliable FIFO messages, if delivered, are always delivered in FIFO order. The second example, shown in Figure 6, consists of four messages sent by process p to process q. Message m2 is lost in transit and never reaches process q. The remaining messages are received by q in the order shown. Message m4 that had overtaken m1 is bu ered on reception as its delivery condition is not satis ed, and its corresponding receive process goes to sleep. When m3 reaches process q it is discarded because end slot has been set to true on the reception of m4 . Message m1 is delivered to q as soon as it arrives. On the delivery of m1, message m4 becomes eligible for delivery because its ST:slot value is now equal to the value of the local slot variable and messages. there are no pending FR It is to be noted that process q did not have to wait for the arrival of m2 before making a decision about discarding m3 even though both the messages are sent using the F R primitive. The logic behind such an implementation is that once an FR message marking the end of a slot is received, its delivery should not be delayed on account of unreliable FIFO messages sent earlier in the slot as loss of these unreliable messages does not adversely a ect the semantics of the slotted-FIFO mode. A delayed F R message is semantically similar to a lost F R message. As F R messages have no ordering constraints they can be delivered whenever 13
they arrive as long as they arrive within the slot. The only messages for which an FR message should wait are the reliable messages sent earlier in the slot, either of type FR (message m5 in the rst example). (message m1 in the second example) or of type FR
6 Applications of Slotted-FIFO Channels Satisfying the reliability and ordering requirements for all messages may entail extensive bu ering, and in some instances long latencies in message delivery [5, 6]. Slotted-FIFO channels provide varying degrees of reliability and ordering for message communication. Such
exibility is desirable for a variety of applications spanning from multimedia to database. For example, several multimedia applications can withstand loss of some messages while still supporting the desired QoS, whereas, long latencies in some message delivery may irremediably degrade the QoS. Slotted-FIFO channels can support such requirements. If the loss of some messages will not adversely a ect the quality of the service, such messages can be sent in an unreliable fashion using the F R or F R primitives. Doing so has two advantages: (i) substantial reduction in the required bu er space at the receiver process and (ii) short message latencies. Indeed, the destination process does not have to bu er the reliable messages that overtake the unreliable messages to ensure the delivery of the latter, and a message that cannot be lost does not have to await the delivery of unreliable messages. To implement the transport layer of a group communication system, an approach similar to slotted-FIFO channels has been proposed in [7] where a message bunch (a set of messages) is wrapped with two reliable messages. Some speci c examples of possible applications of slotted-FIFO channels are as follows. Video telephony: In video telephony the audio and video data streams may be sent along di erent channels; typically high bandwidth channels for video signals and low bandwidth channels for audio signals. It is to be noted that demands are being placed on the audio and video channels by other applications executing concurrently in the network. Hence, the probability distribution of the propagation times of the two di erent data streams between 14
any source-destination pair may be di erent, depending on the loads on the audio and video channels in the network [12]. Ideally, the corresponding audio packet and video frame should be played simultaneously at the destination. However, such synchronization for every audio packet - video frame pair would require very little variability between the latencies of the audio and video channels, and/or extensive bu ering at the receiver. As such requirements are expensive to meet, the following strategy can be employed. Periodically, corresponding audio packets and video frames are sent using the FR primitive, and their delivery to the receiver process is synchronized. The time interval between successive synchronization points corresponds to a slot. During a slot the audio and video signals can be sent using the F R and/or F R primitives. This is because an occasional loss of a small number of audio packets and video frames are beyond human perception. The slot duration should be determined based on the characteristics of the audio and video channels so that the audio and video streams do not get signi cantly out of step during a slot. MPEG video transmission: MPEG compressed video consists of three kinds of frames [8]. The Intraframe frames have no dependencies and can be decoded independently of other frames. A Predicted frame can be decoded only if the previous frame is available. A Bidirectional frame requires the closest Intraframe or Predicted frame before and after it for decoding. As the loss of an Intraframe frame renders all the dependent Predicted frames that follow it useless, the Intraframe frames should be sent using the FR primitive. As loss of a few of the following Predicted frames leads to a marginal degradation in the quality of service, such frames can be sent using F R primitive. As the decoding of Predicted frames is dependent on the decoding of the preceding Intraframe and Predicted frames, employing and F R primitives may lead to non-FIFO delivery of the Predicted frames. Such the FR non-FIFO delivery may not only lead to a degradation in the quality of service, but also unpredictable logical errors in the decoding of the Predicted frames. Sliding window protocol: Enforcing FIFO order among the acknowledgments in a sliding window protocol is both expensive and unnecessary. An acknowledgment for a later packet implicitly acknowledges the reception of earlier packets at the receiver. Hence, to minimize 15
the cost of acknowledgments they can be sent using the F R primitive. If loss of acknowledg primitive. When acknowledgments ments is not acceptable, they can be sent using the FR for earlier packets are received by the sender after the acknowledgments for later packets, such acknowledgments are simply ignored. In doing so, we match the basic idea of the TCP/IP sliding window protocol [10].
7 Conclusion Reliable FIFO message communication has received great attention in the past. This is primarily because database applications have been the dominant network applications so far, and these applications generally require reliable FIFO communication. New multimedia applications have di erent quality of service requirements from database applications, i.e., loss of some packets and occasional non-FIFO delivery is acceptable. Also, multimedia applications require real-time message delivery implying that message latency should be low and message bu ering should be minimized. Hence, communication protocols for database applications are not suitable as well as quite expensive for multimedia applications. In this paper, we presented the slotted-FIFO communication protocol. It supports reliable FIFO, unreliable FIFO, reliable non-FIFO, and unreliable non-FIFO communication modes. The entire communication can be divided into slots demarcated by successive reliable FIFO messages. The reliable messages sent within a slot are always delivered to the destination process after the beginning and before the end of the slot. FIFO messages in a slot, if delivered to the destination, are delivered in the correct order. The delivery of reliable messages is never delayed on account of unreliable messages. This leads to a reduction in message bu ering and latency. Loss of unreliable messages in transit is semantically similar to their late arrival, and requires no special handling. Thus, slotted-FIFO communication is suitable for a variety of distributed applications: real-time as well as non real-time. Also, its simplicity implies that it should be easy to implement. The bookkeeping overheads are also low as each slotted-FIFO channel needs a 16
small, constant number of variables to maintain the message order information.
Acknowledgments The authors would like to thank Roy Friedman of Cornell University for useful comments and discussions on an earlier version of the paper.
References [1] M. Ahuja. Flush Primitives for Asynchronous Distributed System. Information Processing Letters, 34(1):5{12, February 1990. [2] M. Ahuja. An implementation of F-channels. IEEE Transactions on Parallel and Distributed Systems, 658{667, June 1993. [3] R. Baldoni, A. Mostefaoui, and M. Raynal, Causal Delivery of Messages with RealTime Data in Unreliable Networks. Journal of Real-time Systems, Vol 10, no. 3, 245{262, 1996. [4] R. Baldoni, R. Prakash, M. Raynal, and M. Singhal. Ecient -Causal Broadcasting. International Journal of Computer Systems Science and Engineering (to appear), 1996. [5] K. Birman and T. Joseph. Reliable Communication in Presence of Failures. ACM Transactions on Computer Systems, 5(1):47{76, February 1987. [6] K. Birman, A. Schiper, and P. Stephenson. Lightweight Causal and Atomic Broadcast. ACM Transactions on Computer Systems, 9(3):272{314, 1991. [7] G. V. Clocker, N. Huleihel, I. Keidar, D. Dolev. Multimedia multicast transport service for groupware. Proceedings of the TINA'96 Conference, 43{54, VDE Verlag, 1996. [8] D. Gall. MPEG: A video compression standard for multimedia applications. Communications of the ACM, 34(4):46{58, April 1991. 17
[9] T. Houdoin and D. Bonjour. ATM and AAL layer issues concerning multimedia applications. Annals of Telecommunications, 49(5):230{240, 1994. [10] A.S. Tanenbaum. Computer Network. Prentice-Hall International Editions, 1989. [11] R. Yavatkar. MCP: A Protocol for Coordination and Temporal Synchronization in Multimedia Collaborative Applications. In Proceedings of the 12th IEEE International Conference on Distributed Computing Systems, 606{613. IEEE Press, 1992. [12] I. Wakeman. Packetized video: options for interaction between the user, the network and the codec. The Computer Journal, 36(1):55{66, 1993.
18
