An Efficient Membership Management Scheme for Gossip-Based

An Efficient Membership Management Scheme for Gossip-Based Overlay Multicast Yong Sun, Xiangming Wen Institute of Communication Networks Integrated Technique Beijing University of Posts and Telecommunications, Beijing, China [email protected], [email protected]

Abstract  An efficient membership management protocol is proposed named Gomcast (Gossip-based Overlay Multicast). Most traditional gossip solutions work well in small-scale settings, but suffer drastic reduction in reliability and performance at larger scales for reasons of global knowledge required, high overhead and lack of flexibility. Several mechanisms are exploited in this paper to provide node a stable partial view: 1) growing model following power-law distribution with preferential connectivity 2) push-pull gossip mechanism and 3) a DHT-based heartbeat optimization. The experiment results show that the proposed scheme has better performance in large scale. Keywords  gossip, membership management, push-pull, scale-free, overlay multicast.

1. Introduction Traditional IP multicast has many barriers against its deployment, full router dependency, and weak scalability. Overlay multicast (also called application layer multicast, ALM) has been suggested to cope with these problems of IP multicast [1-7]. A virtual infrastructure is built to form an overlay network over IP network topology, and data dissemination is achieved by packet relay in application layer rather than network layer. In application layer multicast system, participating users join and leave the on-going session at will. Compared with conventional multicast, the membership management load is handled non-centrally and unpredictable. These make it more difficult to keep multicast network reliable and efficiency. Therefore, a scalable and reliable group membership management mechanism is necessary due to the highly dynamic and distributive nature of the overlay network. Gossip (or epidemic-style, probabilistic) is a distributed algorithm, which is used in information dissemination, aggregation, overlay topology management, synchronization. The adaptive and self-organization nature enables it to confine the control overhead to local ranges, and adapt to the ever-changing network traffic conditions and group membership. In this way, membership management related overhead is disseminated and tuned automatically to reflect the changes within the network. In gossip-style network, each node keeps partial information about others, and gossips the information to other neighbors. Random nature of gossip algorithm ensures that it can cope with random failures. Mathematical analysis and experiment

results indicate that more than 96% of the nodes can work properly even under very high network dynamics [1]. However, popular implementations of epidemic-style dissemination suffer from three major drawbacks: 1) Global knowledge: They rely on each peer have knowledge of the global membership. 2) Network overhead: A high overhead is imposed when applied to wide-area settings. 3) Lack of flexible adaptability: They impose the same load on process group members selected uniformly at random (flatly) from the set of all nodes. Traditional gossip solutions to the problem work well in small-scale settings, but suffer drastic reduction in reliability and performance at larger scales. Massive P2P video/audio streaming propagating in large networks needs a stable distributed membership management which can provide each node a partial view instead of a global knowledge of all nodes in system. In this paper, a protocol named Gomcast (gossip-based overlay multicast) is proposed to cope with the problem of large-scale multicast application in overlay network. The contributions of this paper mainly contain: 1) Growing model follows power-law distribution is proposed with preferential connectivity using some self-adapted metric. 2) Push-pull mechanism for quickly convergence is used to reduce the propagated latency. and 3) A DHT-based heartbeat is used as optimization scheme to avoid node inefficacy. The rest of the paper is organized as follows. Section 2 introduces some related work about overlay multicast and gossip-based membership management. Section 3 describes the details of the protocol. Section 4 describes the experimental setup. Simulation results are also provided. And section 5 concludes our work.

2. Related Work 2.1 Overlay Multicast Many schemes are proposed to ensure scalable and reliable in overlay multicast. Overlay multicast (also named ALM) can be divided into structured and unstructured system. Several of them are well known. A well-known unstructured system is Gnutella [7] which is used to share files. In Gnutella system, the query message for locating data items is flooding over the overlay network build by the join nodes. Gnutella is criticized for being non-scalable

This work was supported by the National Natural Science Foundation of China under Grant No.60743007 & BUPT Education Foundation.

ISBN 978-89-5519-136-3

-981-

Feb. 17-20, 2008 ICACT 2008

and not suitable for large-scale P2P networks. Chord [2] is a famous structured P2P system based on Distributed Hash Tables (DHT). In Chord system, a node ID ring is organized with all nodes in the overlay network. The delivery of a query message to destination can be guaranteed within O(log N ) hops. Although the advantage of scalability, fault-tolerance and load balance for maintaining the information related shared resources, long-latency delay may exist in DHT-based P2P networks. Also for reasons of specified node ID, Chord is not adaptability for large-scale application. Other hybrid schemes are proposed, e.g., PAM [4]. PAM is proposed for hierarchical overlay multicast based on host group model and IP topology awareness. The protocol uses traditional IP topology information by IGMP or MLD protocol to get rid of inefficiency of application level routing. 2.2 Gossip-based Membership Management Recently, gossip-based protocols has got more attention by reason of its adaptive and nature scalable. Gossip-based protocols spread multicasts in a group with a randomized P2P fashion much like the spread of rumor in society, or of a contagious disease in a population. SCAMP [1] is a P2P membership protocol which operates in a fully decentralized manner and provides each member with a partial view of the group membership. The protocol is self-organizing in the sense that the size of partial views which is a function of the group size. [6] provides a theoretical analysis of gossip-based protocols which relates their reliability to key system parameters (system size, failure rates, and number of gossip targets). The research results show that providing each peer with only a small subset of the total membership information and organizing members into a hierarchical structure will reduce the load on the network. A hybrid protocol Rpbcast [8] was proposed, which consists of three phases: the first phase uses an unreliable IP multicast. During the second phase, a gossiping step is initiated. If it fails, a third deterministic phase using loggers is invoked. An adaptive algorithm is proposed in [9], which captures the changes of the network and adjusts the parameter settings dynamically, bringing adaptability and reducing the overhead. Some practicality systems are developed. Coolstreaming [3] realize an efficient system using gossip to find peers. Gridmedia [5] was experimented to relay Spring Festival Evening of CCTV in China, the peak value of online people number on the same time is over 15000.

3. Gossip-Based overlay Multicast Protocol The mechanism of gossip algorithms is that schemes network to distribute the propagation burden and in which a node communicates with a randomly chosen neighbor. There are not a centralized entity for facilitating dissemination, communication and time-synchronization. The network topology may not be completely known to the nodes of the

ISBN 978-89-5519-136-3

-982-

network. Also, topology may change continuously as new node join and old nodes leave the network. Membership management is the primary mechanism in the protocol. Peer has been developed to cope with the properties and thus to reach extreme scalability and support dynamism. Gossip makes it possible to achieve extreme scalability and performance at the cost of some redundancy. This results in probabilistic behavior of the reliability of multicast delivery at recipients, and per-process overhead and latency rise slowly (logarithmically) with group size. Reliability of gossip is achieved by introducing sufficient redundancy by making the number of gossip targets chosen by each member large enough, as a function of the group size. Mathematical theory of epidemics has proved that the number of gossip targets to the fraction of group members who eventually receive the gossip message (this is equal to the probability that an arbitrary group member will receive the message) [6]. If there are n nodes and each node gossips to log(n) + k other nodes on average, then the probability that −k

everyone gets the message converges to e − e . This refers to the probability that every node receives the message. 3.1 Preferential connectivity in Scale-free network Research has proved that uniform random topology is not practical, but grows with power-law distribution. Uniform randomness has long been generally assumed based only on (wrong) intuition. Previous theoretical results with assumption need be revised to properly describe the observed behavior. Authors in [10] prove that few instance leads to a uniform sampling, rendering traditional theoretical approaches invalid when those protocols are applied as a sampling service. Complex networks theory investigates phenomena in nature, biology, sociology, and computer science that have a so-called “small-world” topology. Internet has the attribute of scale-free and follows power-law distribution. The scale-free property of the network is in favor of the information dissemination, especially if those nodes with higher degree, or named as hubs, are reached by the gossip message, and the gossip message will propagate through the network at a very high speed. In traditional gossip algorithm, each node choose a fixed number of members, uniformly at random from its partial membership list, and each gossip round also is a fixed time interval. In SCAMP, the partial views is provided to each node are chosen uniformly among all group members. Random view selection results in an unbalanced distribution and slow convergence. Some preferentially chosen of partial view is needed. A pseudo-random way by choosing the targets preferentially is defined under the so-called preferential connectivity (PC) model as follows, according to some metric. We consider models where nodes are grouped into clusters and partial views are largely restricted to nodes within the same cluster. This can be extended to hierarchical models with an arbitrary number of levels. A metric is used as B m =C× 2 (1) D

Feb. 17-20, 2008 ICACT 2008

Where C is a constant, B stands for residual bandwidth of the node, and D is the delay between the nodes, or simply network distance. With the metric, a view list is kept by every node ordered by metric. In the list, particular view can be chosen from the first or last k elements of the neighbor view. We call it head or tail, which means the priority of nodes selection implicitly. The view selection now is not uniform random, but preferential connected. Note that peer selection is still selected randomly. The metric also adapt the length of the gossip round and the scope of the gossip targets selection. In the T time slot, node i contacts some neighboring nodes selected from view list with probability Pi .Both T and Pi are fixed in traditional gossip algorithm, we tune them in terms of time domain and space domain. More detail description can be found in table 1. The above improvement changes the same load on process of group members selection and gossip round interval, which leads to more scalable and flexible adaptable without sacrificing reliability. 3.2 push-pull mechanism Gossip is the primary mechanism of message propagating and synchronizing in the protocol. When node receives packet, he pushes it to all nodes in his partial view. On the other hand, if a node lacks some packets in his multicast process, he will ask a gossiper to transmit the packet (gossip-pull), or wait for some gossiper propagating it (gossip-push), or both. SCAMP[1] is a pure push algorithm. Figure 1 shows the two forms of gossip propagation. In (1), node A pushes out a message to node B, and tells B that he has packet X, Y and Z. Node B checks his own data packets list, and finds lacking of packet X, after a message back to A, A pushes packet X to B. In (2), node C pulls packet Y from D. Firstly, C sends a message with lacking of packet Y and having packet X, Z. D gets the message, and transmits Y to C. The latter model has only 2 steps to transact, and less overhead is proceeding. Also, under normal conditions, where a multicast reaches most receivers, gossip-pull will have lower expected delivery latency than gossip-push. The most advancement of pull is that it allows a receiver to request transmission immediately after detecting a lacking packet instead of waiting until the next gossip cycle, thus further improving packet delivery latency. Some protocols use only push or pull. If only push, it has long delivery latency, more unstable, and converges slowly in the growing overlay scenario, while can guarantee reliable

Table 1. Pseudo Code of Gossip Membership Management Protocol Thread 1 active (initiate communication with other nodes) do forever wait ( T time slots) //T is the gossip interval time. pi = Ei ∑ Ei

ni ← selectPeer( pi )

//peer selection

if push then buffer ← merge (view, {mydescriptor}) //merge local node ID with view send buffer to ni //view propagation else send {packet} to ni //trigger response if pull then request viewn from ni buffer ← merge (view,viewn) view ← selectView(buffer) //view selection T ′ = T (∑ E N ) E Thread 2 passive (wait incoming message and handle a forward) do forever if pull then buffer ← merge (view, {mydescriptor}) send buffer to n j send {packet} to n j buffer ← merge (view,viewn) view ← selectView(buffer)

dissemination. If only pull, it will lead to severe central clustering which would not handle dynamic nodes joining at will and may converge to a star topology, and it is undesirable. In a word, push is suitable for few source and many applicants when is the initial stages of multicast, and pull adapts to the scenario of much source and some askers, where source may change from applicants. Combining push with pull is a better scheme which uses advancements both. The performance of push-pull is clearly superior compared to push-only approaches. In large scale network, listing missing message in gossip-pull also usually be small than describing the entire message buffer in gossip-push, especially under high send rates, Two threads are planed to accomplish the goal. Thread 1 initiates communication with other nodes, which is called active-thread. Thread 2 monitors incoming message and handles the forwarding, which is called passive-thread. Three dimensions are contained in the design of gossip-based membership management protocol: peer selection, view selection, view propagation. Every node maintains a membership table, named partial view, the size of which grows with the size of the system. Each node selects a peer periodically to exchange membership information with. View selection means that peers truncate their views in order, and select a new subset of partial view. Pseudo code is showed in table 1. 3.3 Optimization Some optimization methods are applied to improve the performance. Certain information about one or several on-tree nodes will be provided, by some out-of-band mechanism like logger or bootstrap, to the newcomers. Some ideas of what the current multicast rate is and the magnitude of delay between the nodes can be measured with timestamps of the received packets or with the ping program.

Figure 1. Gossip View Propagation

ISBN 978-89-5519-136-3

-983-

Feb. 17-20, 2008 ICACT 2008

Also, either node failure or leave can cause the network to become disconnected. The disconnection of larger subsets has vanishingly small probability as the system size N grows large. The node becomes isolated from the graph when all nodes containing its identifier in their partial views have failed. In order to reconnect such nodes, a heartbeat mechanism based on DHT is proposed. Each node periodically sends heartbeat messages to the nodes in its partial view (these are not notifications and are not propagated any further). A node that hasn’t received a heartbeat message in a long time knows that it is isolated and resubscribes through an arbitrary node in its partial view. Let Dn be the last known sequence number of data packets, then Hn = HASH (∪ Dn) using a collision-free hash function HASH .Instead of gossiping all data packets numbers, only hash signature is propagated. If the hash signature is same, no further information exchange is necessary. Else, means the disagreements is occurring, and a new gossip-pull may start immediately. The optimization of heartbeat makes network connected, also for reasons of only the hash signature checkout, gossip-pull will startup quickly, which improves latency farther, reduces the overhead to propagate and makes the performance much better.

Figure 3. Compare of Uniform Random and Preferential Connectivity

4. Simulation Experiments

follow the power-law degree distribution, while ER model (uniform random) [13] does not fit the according line which is the probability curve of P( k ) = 2m 2 k −3 . Note that coordinates in figure 3 is double logarithmically, and nodes number is 10000. Figure 4 shows the average path length of uniform random and growing model. In the figure, growing model converges to a stable state more quickly than uniform random model. In the end, both of models aggregate the average path length to about 2.8. We also observe that overlay properties can converge to the same structure with a low average path length even if the

Simulation is constructed based on PeerSim simulator [11]. The topology constructed method with scale-free network using preferential connectivity is used to model the large-scale multicast disseminated application which follows power-law distributions. Figure 2 is a topology of 350 nodes. In the network, the node with largest degree 47 is in the center of the figure, and degree of others nodes lies betweens 1-5. Most of experiments are made with the topology of 350 nodes and 10000 nodes. In figure 2, we can find clearly that small-world exists. On the edge of network, some cluster occurs. The average degree distribute of topology is 3, which fits the result of the Internet measure. We also measure the extent of topologies deviate from the uniform random model mentioned earlier. As in figure 3, both our PC (preferential connectivity) model and BA model [12]

Figure 4. Average Path Length of Random and Growing Model

Figure 2. Network Topology of 350 Nodes

Figure 5. Clustering Coefficient of Random and Growing Model

ISBN 978-89-5519-136-3

-984-

Feb. 17-20, 2008 ICACT 2008

group members have a message. The performance of push-pull is superior compared to push-only approaches clearly.

5. Conclusions In this paper, an efficient membership management protocol named Gomcast is proposed. Three mechanisms are imposed to improve the performance of overlay multicast in large scale. Growing model follows power-law distribution is proposed with preferential connectivity using some self-adapted metric. Push-pull mechanism for quickly convergence is used to reduce propagated latency. A DHT-based heartbeat is used as optimization scheme to avoid node inefficacy. Simulation and experiments show that the scheme is effective. The following job is to perfect the simulation, and a practical system will be realized in the future.

Figure 6. Overhead of Gomcast, Gridmedia and Coolstreaming

REFERENCES

Figure 7. Dynamics Link Stress of Gomcast and SCAMP

initial topology is highly unstructured or has a large diameter length. Figure 5 shows the clustering coefficient of uniform random and growing model. The clustering coefficient of a node a is defined as the number of edges between the neighbors of a divided by the number of all possible edges between those neighbors. Random model has higher clustering coefficient at beginning than growing model, but slowly converge to a close value. Also, we have observed that the network load can be reduced by clustering on the basis of proximity in the network. In order to evaluate the gossip message overhead, we compare two practicality systems (Coolstreaming, Gridmedia) with Gomcast. Figure 6 shows the control overhead in the membership management process. All of systems have an overhead only between 1%- 2% in the total traffic overhead. This means that gossip protocol is lightweight. Consider the scenario where a multicast packet is received by a fraction of 10000 receivers, Figure 7 shows the dynamics link stress of two different systems with the lifetime of nodes. Gomcast which uses push-pull scheme has better stability and less stress than SCAMP which use only push scheme. The partitioning of the push version of the protocols is due to the fact that only some first, central node will distribute new links to all new members. Gossip pull is fast convergence and low latency and is more effective than push when a majority of the

ISBN 978-89-5519-136-3

-985-

[1] A. J. Ganesh, A.-M. Kermarrec, and L. Massoulie, "Peer-to-peer membership management for gossip-based protocols," IEEE Transactions on Computers, vol. 52, pp. 139-149, 2003. [2] I. Stoica, R. Morris, D. Liben-Nowell, D. R. Karger, M. F. Kaashoek, F. Dabek, and H. Balakrishnan, "Chord: A scalable peer-to-peer lookup protocol for Internet applications," IEEE/ACM Transactions on Networking, vol. 11, pp. 17-32, 2003. [3] X. Zhang, J. Liu, B. Li, and T.-S. P. Yum, "CoolStreaming/DONet: A data-driven overlay network for peer-to-peer live media streaming," Proceedings - IEEE INFOCOM, Miami, FL, United States, 2005, pp. 2102-2111. [4] D.-K. Kim, K.-I. Kim, I.-S. Hwang, and S.-H. Kim, "Hierarchical overlay multicast based on host group model and topology-awareness," The 7th International Conference on Advanced Communication Technology, ICACT 2005, Phoenix Park, South Korea, 2005, pp. 335-339. [5] J.-G. Luo, M. Zhang, L. Zhao, and S.-Q. Yang, "A large-scale live video streaming system based on P2P networks," Ruan Jian Xue Bao/Journal of Software, vol. 18, pp. 391-399, 2007. [6] A. M. Kermarrec, L. Massoulie, and A. J. Ganesh, "Probabilistic reliable dissemination in large-scale systems," Parallel and Distributed Systems, IEEE Transactions on, vol. 14, pp. 248-258, 2003. [7] Gnutella, http://www.gnutella.com. [8] S. Qixiang and D. C. Sturman, "A gossip-based reliable multicast for large-scale high-throughput applications," in Proceedings International Conference on Dependable Systems and Networks(DSN 2000) 2000, pp. 347-358. [9] R. Bin, I. Khalil, and Z. Tari, "An Adaptive Membership Algorithm for Application Layer Multicast," in Networking and Services, 2006. ICNS '06. International conference on, 2006, pp. 35-35. [10] M. Jelasity, R. Guerraoui, A.-M. Kermarrec, and M. v. Steen, "The peer sampling service: Experimental evaluation of unstructured gossip-based implementations," in Middleware 2004,volume 3231 of Lecture Notes in Computer Science,Springer-Verlag, 2004, pp. 79-98. [11] PeerSim, http://peersim.sourceforge.net/. [12] R. Albert and A. L. Barabási, "Statistical mechanics of complex networks," Reviews of Modern Physics, vol. 74, pp. 47-97, 2002. [13] P. Erdös and A. Rényi, "On the evolution of random graphs," Publication of the Mathematical Institute of the Hungarian Academy of Sciences, vol. 5, pp. 17-61, 1960.

Feb. 17-20, 2008 ICACT 2008