Toward Interoperable Data Synchronization with ... - Semantic Scholar

2 downloads 0 Views 137KB Size Report
Apratim Purakayastha, Dennis Shea, Josephine Cheng, Don. Clare, Gerry Fisher, Rob Scott, Yudong Sun, Wone, May, and. Quinton Zondervan. Enterprise data ...
To be presented at 3rd IEEE Workshop on Mobile Computing Systems and Applications (WMCSA 2000), Monterey, California, December 7-8, 2000. Copyright © 2000 by IEEE. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee.

Toward Interoperable Data Synchronization with COSMOS Norman H. Cohen Apratim Purakayastha IBM Thomas J. Watson Research Center {ncohen,apu}@us.ibm.com Abstract COSMOS is a method for specifying the behavior of a synchronizable data store by modeling the data store as a state machine. Such a model can be used to define a standard for interoperable data synchronization systems, defining the sequence of messages sent and received during synchronization without constraining the implementation of a conforming system. COSMOS models can be collected, studied, and categorized, allowing a synchronization scheme with the properties appropriate for a given application to be selected from a catalog. Alternatively, for preexisting synchronization systems built without a common standard, COSMOS models can serve as a specification for an adapter allowing the systems to interoperate.

1. Introduction Applications running on mobile devices often share data with remote computers. Since a mobile device may be disconnected or weakly connected, mobile devices typically maintain local copies of the shared data, and synchronize them from time to time with remote copies. A wide variety of mobile data-synchronization systems have been developed, including Coda [1], Ficus [2,3], Bayou [4], Lotus Notes [5], PalmPilot HotSync [6], and the datasynchronization framework of the Mobile Network Computing Reference Specification (MNCRS) [7]. Synchronization systems such as these can differ in many ways, the most important of which are enumerated in Section 1.1. Despite the variations, each of these datasynchronization systems can be described by an approach we call Co-Operative State Machines for Object Synchronization, or COSMOS. COSMOS is an abstract model of a data-synchronization system. Each data store is modeled as a state machine. A COSMOS model captures the externally visible behavior of a synchronization system, avoiding details of programming language, implementation techniques, and the application

interface for querying and updating the local copy of a store. Thus the model can serve as a formal specification of a synchronizable store, allowing independently implemented stores conforming to the specification to interoperate. Indeed, COSMOS provides a framework for classifying synchronization systems based on particular attributes of the model, so that independent parties interested in developing interoperable synchronization systems can choose a model from a catalog. A COSMOS model also serves as a basis for proving correctness and other properties. Finally a COSMOS model can provide a basis for the automatic generation of test cases. COSMOS can be used to model data synchronization at various levels of detail. A more abstract model elucidates the fundamental strategies of the synchronization scheme, and simplifies reasoning about the system. A more detailed model can address issues such as authentication and recovery from communication errors, and can serve as an implementation specification.

1.1. Variations among synchronization systems A wide variety of systems, with very different goals and constraints, synchronize collections of data on intermittently connected platforms. Among the most important attributes that distinguish these systems are: Ÿ The nature of the data synchronized. Coda and Ficus synchronize files. Bayou synchronizes tuples. Lotus Notes synchronizes semistructured documents. MNCRS synchronizes Java objects. Ÿ The correspondence among synchronized items. In some systems, all replicas of a synchronized item have the same form. In contrast, in the MDSS system [8], each Java object on a client device corresponds to a relational-database record on the server. Lotus Notes synchronization can be configured to send only part of a document to a mobile device. HotSync synchronizes PalmPilot database records with addressbook, calendar, and other entries of desktop applica-

tions; the PalmPilot records and the desktop entries may have substantially different sets of attributes. Ÿ The correspondence among synchronized collections. Complete synchronization of two Bayou or MNCRS stores causes the stores to contain the same set of items. MDSS is an example of a system in which each client contains items corresponding to a subset of the items in a central store. PalmPilot HotSync allows for an item to be removed from one device and for the corresponding item to be archived on the other device. Ÿ Management of unreliable communications. Some systems delegate the management of communications to an underlying layer that is presumed to provide reliable, in-order delivery of all messages. Other systems manage communications themselves, and specify the states resulting from various kinds of failures. Ÿ Synchronization topology. Synchronization topology is defined not by the network communication paths among devices containing data stores, but by which stores can synchronize with which. Five synchronization topologies, in order of increasing restrictiveness, are: ú peer-to-peer systems like Bayou, Ficus, and MNCRS, in which a store may synchronize with any corresponding store ú indirect hierarchies, in which corresponding stores form a tree, and a store may synchronize with any of its ancestors or descendants ú direct hierarchies, in which corresponding stores form a tree, and a store may synchronize with its parent or immediate children ú star topologies, consisting of one central store and some number of subordinate stores, each of which may synchronize with only with the central store ú a dedicated-pair topology, consisting of two stores that synchronize only with each other, as in the case of a single handheld device that is synchronized with a single desktop computer Ÿ Concurrency control. Some synchronization systems, including MNCRS, support asynchronous protocols, in which independent streams of updates are sent in each direction, without coordination. Such systems could be synchronized, for example, by e-mail messages. Other systems rely on a requestresponse protocol in which two stores being synchronized participate in a rigorously structured dialog. Some systems, such as MNCRS, allow synchronizations with multiple stores to be in progress at the same time. Other systems, such as MDSS and HotSync, disallow any modifications to a store while it is being synchronized, either by another synchronization session or by an application. Ceri, Houtsma, Keller, and Samarati [9] enumerate additional distinctions among data-synchronization systems.

1.2. Elements of COSMOS A synchronization system can be characterized by a state-machine model of a data store, one or more protocols for communication among data stores, one or more transport mechanisms, and perhaps a network-wide mechanism or policy (such as adaptive rings [2] or epidemic algorithms [10]) for initiating synchronization between two data stores. The state of the machine reflects both the stored data and bookkeeping information used for synchronization. Each change to either the stored data or the bookkeeping information represents a transition from one state to another. The stored data is modeled as a map from keys to entities that include data values. The nature of the bookkeeping information varies from one COSMOS model to another. However, the state is always assumed to include a fixed replica ID uniquely distinguishing a data store from all corresponding data stores. A protocol comprises both a notation for messages and a set of rules for determining the times at which various forms of messages are sent. The state of a COSMOS state machine determines the outgoing messages that may be generated. Both the generation of outgoing messages and the receipt of incoming messages trigger state transitions. A state transition may also result from an update to the stored data by an application program using the data store. We call such an update a local mutation. A local mutation is modeled as a pair consisting of a key and a value. The value may be a data value or, in some models, a special token such as delete or archive. A COSMOS model does not specify the nature of the keys and the data values. Keys might be file pathnames, relational-database keys, or object IDs, for example; data values might be files, relational-database tuples, or objects. Similarly, COSMOS does not specify the nature of replica IDs or the mechanism by which they are assigned. COSMOS does not model the API through which an application performs local mutations or retrieves the data value (if any) to which a key is currently mapped. In some COSMOS models, conflicting updates cause a reconciliation mechanism (such as a Coda applicationspecific resolver, a Bayou merge procedure, or a method of an MNCRS stored object) to be invoked to determine the resulting data value. This mechanism is modeled as a function reconcile that takes the two conflicting updates as arguments and evaluates to the reconciled value. The COSMOS model does not specify whether the reconcile function represents, for example, a fixed policy, a single application-provided algorithm, or an algorithm associated with the individual data values involved. A single COSMOS data model can model all nodes in a network of interoperating data stores with radically different reconciliation mechanisms.

COSMOS models the abstract behavior of a data store, not its implementation. A COSMOS state model is built out of high-level mathematical collections such as sequences, sets, and maps, making it easy to specify state transitions in a mostly declarative style. There will be many ways for an implementation to represent this abstract state, and many algorithms effecting the specified state transitions.

1.3. Structure of this paper In Section 2.2, we describe and illustrate the specifics of COSMOS models. Section 3 discusses the role of COSMOS in enabling data-synchronization systems to interoperate. We discuss related work in Section 4. We conclude in Section 5 with a discussion of the origins and future directions of our COSMOS work.

2. Illustrations of the COSMOS formalism To introduce the COSMOS modeling technique, we present the specification of a simplistic synchronization system in Section 2.2. Section 2.3 presents a more interesting, but more intricate example, PalmPilot HotSync. Section 2.1 explains the Cosmos notations that are used in both examples.

2.1. Notation COSMOS models are written in an expressive mathematical notation loosely inspired by Z [11, 12], SETL 2 [13], and SEDL [14]. The following symbols represent fundamental sets: R is the set of replica IDs, K is the set of keys, D is the set of data values, B is the set of boolean values, and N is the set of nonnegative integers. If S is a set, arb S denotes an arbitrary member of S. For the set K, the special notation new K denotes a newly generated key, distinct from all previously generated keys. The expression { tx for xcS | Px } denotes the set containing the value tx for each value x in S for which Px is true; we write {xcS | Px} as a shorthand for {x for xcS | Px}. (The symbol x is an iteration variable.) If S1 and S2 are sets, S1-S2 is the set consisting of all elements of S1 that are not elements of S2. If S1 and S2 are sets, (S1→S2) denotes the set of mappings from S1 to S2. Each such mapping is a set of pairs x→y consisting of a first component x in S1 and a second component y in S2, such that no two pairs in the set have the same first component. If f is an element of (S1→S2) and x is an element of S1, then the expression f(x) denotes the second component of the pair (if any) in f whose first component is x, and dom f denotes the set of first components of pairs in f. If M is a mapping, we may describe a set with an expression of the form

{ txy for x→y c M | Pxy } (shorthand {x→ycM | Pxy}); the form x→y acts as an iteration variable iterating over each pair in M, with x bound to the first component and y bound to the second component of each pair. S* denotes the set of sequences whose elements belong to the set S. We may construct a sequence by enumerating the elements in order, as in [x1,...,xn], by iterating over some other sequence s in order and selecting the elements satisfying some predicate, as in [tx for x in s | Px] (shorthand [x in s | Px]), or by iterating over the elements of a set in an unspecified order and selecting the elements satisfying some predicate, as in [tx for xcs | Px] (shorthand [ xcs | Px ]) where s is a set. The expression s1*s2 denotes the concatenation of sequences s1 and s2. If s is a nonempty sequence, head s denotes the first element of the sequence and tail s denotes the subsequence consisting of all but the first element. The notation T: ßn1: T1; ...; nk: Tk à declares T to be the Cartesian product T1%...%Tk , i.e., the set of all k-tuples whose ith component belongs to Ti for 1[i[k. The symbol ni is a name given to the ith component of such a k-tuple for readability. If x is an element of T, then x.ni denotes the ith component of that tuple. A value in T can be denoted by the expression ßn1: v1; ...; nk: vk à where vi is an expression of type Ti. The symbol / is the overriding operator. It may be applied to tuples, maps, or sequences. Informally, it denotes the value identical to its left operand except where overridden as specified by its right operand. If m1 and m2 are mappings, then (m1/m2)(x) = if xcdom m2 then m2(x) else m1(x) fi. If s1 and s2 are sequences, s1 / s2 = [ x in s1 | ~(x in s2) ] * s2. (Thus, if x occurs once in sequence s, s/[x] is identical to s, but with x moved to the end.) For a tuple t of type ßn1: T1; ...; nk: Tkà, if {m1,...,mj}`{n1,...,nk}, the expression t/ßm1: v1; ...; mj: vj à denotes the tuple all of whose components are equal to the corresponding components of t, except for the components named m1,...,mj, which have values v1,...,vj, respectively. Complex expressions can be decomposed into digestible pieces, and common subexpressions eliminated, through expressions of the form let v1 = t1, ..., vn = tn in tn+1. The variable vi, 1[i[n, stands for the value of the expression ti in each of the expressions tj, i