A Multiplayer Games Priority Selection Strategy for Use in QoS-Aware Networks Brian Carrig1, David Denieffe1 and John Murphy2 School of Engineering, Institute of Technology Carlow, Kilkenny Road, Carlow, Ireland. Tel: 059 9170597, Fax: 059 9170517, email:
[email protected] 2 Department of Computer Science and Informatics, University College Dublin, Belfield, Dublin 4, Ireland, email:
[email protected] 1
Abstract Future Quality of Service (QoS) aware networks, such as those based on the Differentiated Services (DiffServ) architecture, will provide users with the opportunity to prioritize the traffic they receive and generate. Different priorities will entail different per-byte or per-packet charges. To aid users operating within such an environment, a priority selection strategy is required. In this paper, we propose a priority selection strategy based on Consumer Surplus (CS), which is the difference between what the user is willing to pay (utility) and the actual cost. We then evaluate this strategy in a networking scenario where a user is involved in a multiplayer Counterstrike game, in which different priorities can be assigned to the downstream traffic. A utility curve, applicable to a First Person Shooter (FPS) game like Counterstrike, is presented. Using simulation results we show that for varied traffic loads, the CS strategy offers comparable application performance to that of always selecting the highest priority. However it does so at a reduced cost to the user..
1. Introduction Today the “best effort” Internet service model is still predominant, serving the network well since its inception in the early 1970s. However, applications with tight lower bounds on delay, delay jitter and packet loss, such as interactive voice and video or multiplayer gaming can experience difficulties. Over the past decade much research has been carried out on the best way to modify the existing Internet architecture so that it can have an awareness of the Quality of Service (QoS) requirements of individual traffic flows or aggregates. The IETF has developed two major QoS architectures, the Integrated Services (IntServ) [1] architecture and the Differentiated Services (DiffServ) [2] architecture. IntServ provides absolute guarantees for end-to-end QoS through the explicit reservation of resources. However, it requires that all intermediary routers on a path have IntServ capability and that they maintain per-flow state
information. This causes a scalability issue which has led to the development of DiffServ. Instead of classifying on a per-flow basis, DiffServ classifies individual packets into aggregate classes, based on the value of the codepoint in the packet header. While this lends greatly to the scalability of DiffServ, it operates at a much coarser level of granularity and provides only relative QoS assurances. DiffServ would entail many operational changes to the network but unlike IntServ, these changes can be made in an incremental fashion. In DiffServ-based networking environments, it is feasible that the user will be allowed some degree of control regarding the packet-level QoS applied to her traffic, although possibly within the constraints of a particular service profile. If a user is given the freedom to assign a priority to the traffic she generates and receives, then the user will need to formulate a strategy for priority selection. Such a strategy may be as simple as always selecting the best effort service, in order to minimise cost. Or something more complex, performed by an agent, similar to that contained in [5]. Interactive multiplayer networked games are an example of a QoS-sensitive application. Although it can be difficult to correctly identify games traffic on the network [3], it is known that the volume of games traffic on the Internet has been steadily increasing in recent years [6]. The First Person Shooter (FPS), Counterstrike, a modification of the Halflife engine, has over 19.5 million legal owners with an average of over 85,000 simultaneous players playing at any one point in time according to the popular GameSpy website [4]. FPS games, like Counterstrike, are particularly sensitive to network latency. In many cases, such delay can be the difference between winning and losing [7]. Despite the widespread lack of QoS mechanisms on the Internet today and the massive popularity of networked games, surveys such as the one detailed in [8], indicate that gamers are indeed sensitive to the QoS they receive while playing. In this paper, we introduce a novel priority selection strategy based upon the concept of Consumer Surplus (CS). This is a microeconomics term, describing the difference between the utility, or satisfaction, to the user
of a particular good or service and the cost of the good or service. Utility may be viewed as the value of a good to the user – their willingness-to-pay. The surplus between this and the amount they actually pay is the CS. It is generally assumed that users will behave in a manner that maximises their CS. The users monitor the QoS that may be obtained on the different service classes and choose the class that is best suited to their needs. The service class which offers the maximum CS is the one the user chooses to transmit and receive data on. In the next section we provide an overview of related research. This is followed by section 3, which details the proposed CS priority selection strategy. Section 4 defines the testing scenarios and section 5 presents the experiment results, comparing the CS strategy with other applicable strategies. The paper concludes with a discussion of the main contributions of the paper and some ideas for future work.
2. Related work QoS for heterogeneous IP networks has been the subject of much Internet research for the past decade, yet a large portion of IP networks still do not employ any QoS mechanisms. Service providers widely over-provision the backbones of their networks in order to provide adequate QoS to their customers [9]. This practice is likely to continue while core bandwidth remains inexpensive. Two IETF standards have been proposed since the early nineties. These are the Integrated Services (IntServ) architecture and the Differentiated Services (DiffServ) architecture. Both IntServ and DiffServ operate at the packet-level, without concern for the lower levels. However, the deployment of IntServ has been hindered by scalability concerns and slow adoption by both end and intermediary systems. DiffServ on the other hand, has a number of open issues relating to the reservation or allocation of resources and administrative complexity. Several other proposals for service differentiation also exist, such as those covered in [11-13]. With few exceptions, proposals require the use of additional pricing controls to limit abuse, even if these prices are only applied during periods of congestion. Though there is a concentration of work on pricing (see [14] for an overview of the field), much of the focus is on how providers might best optimize their revenue or maximize social welfare. Reference [15], provides an analytical model for service differentiation across a range of queuing disciplines, concluding that Priority Queuing (PQ) is more efficient economically than either the First In, First Out (FIFO) or Generalised Processor Sharing (GPS) queue schedulers. Indeed, the use of a GPS scheduler for service differentiation purposes may result in less revenue being accrued by the provider than in a flat undifferentiated
network. This follows on from work in [16] where a utility function is used to describe the sensitivity of users to network delay. The concept of utility itself is an economics term, adequately described in [10]. It is a measure of the satisfaction derived by the consumer of a particular good or service. The assumption is that rational beings will always behave in a way that maximises their utility, allowing for budgetary constraints. The maximum amount of money a user is willing to exchange for a service can be said to be equal to the utility of that service. It is for this reason that utility is often expressed in the form of a monetary unit. The use of a utility function allows users to directly compare the service level offered by different networks, or indeed, multiple services classes within the one network. In an extensive analysis of a range of popular on-line multiplayer games in [26], the authors observed that the traffic from FPS games is highly predictable. The same characteristics, such as the transmission of large, highly periodic bursts of small UDP packets and the requirement for low-latency point-to-point communication is observed for all games of this class. This is largely due to the synchronous nature of current game designs, the uniform requirement for high levels of interactivity and the phenomenon of narrowest last mile link saturation among players. The finding that this aggregate traffic can be generalised and is in fact almost linear with the number of players involved is supported in [27]. The contribution of the access network to the delay experienced by gamers is discussed in [17]. The delay introduced by the access network is of primary importance to gamers, as they typically choose game servers residing close to their geographical location. Although due to issues such as a disparity between geographical location and network topology, poor server selection mechanisms and a shortage of servers in some areas, this is often not the case. The paper concludes that end-to-end delay is the most important parameter for games traffic and that service differentiation is required if multiple services are to be run over the same last mile link. The sensitivity of Quake 3 gamers to latency is estimated in [19] although the conclusion that most Quake 3 gamers prefer roundtrip times of below 150ms has been vigorously argued against by gamers on the Slashdot forum [20]. In [21], a more empirical approach to measuring the sensitivity of Quake 3 players to delay is taken and although the sample size is small, the results largely concur with [19]. However, in an evaluation of Unreal Tournament, users indicated that roundtrip delays of greater than 60ms were considered “disturbing” by some [25]. In one study of Internet and WAN latency [18], it was discovered that roundtrip times between distributed sites could vary between a few milliseconds to over 2 seconds,
with no significant correlation between geographical distance and the number of hops.
latency,
increase in satisfaction, so the number of measurements taken should be much less than that generated by the application traffic.
3. CS Priority selection strategy This paper considers a scenario where a user involved in a multiplayer Counterstrike game may choose the priority of the downstream traffic received from the game server. The user indicates the priority to be received by marking the upstream traffic to the server. The networking setup is analogous to many residential DSL networks in operation today such as that illustrated in fig. 1.
Di = αDi −1 + (1 − α ) M i
The delay measurements for all classes are inserted into a continuous piecewise linear function, as shown in fig. 2. The utility for the user is considered excellent at roundtrip delays below 50ms. Where the roundtrip delay value for packet i, Di, is less than 50ms, a maximum utility value of Umax or 1.0 is returned. This value decreases linearly for delay values above 50ms and once the delay exceeds 100ms; no utility is obtained by the user. This is captured in (2), where Dmax is the maximum acceptable roundtrip time and Dth is the threshold below which maximum utility is obtained. The use of 50ms and 100ms as values for Dth and Dmax respectively, is speculative rather than authoritative. Its use is intended to represent a delay-sensitive user. Users who are less sensitive to delay may require larger values for Dth and Dmax.
Fig. 1. DSL Network Components (from DSL Forum)
Utility Function 1.2 1 0.8 Utility
The user may deploy a number of priority selection strategies to maximise her satisfaction. For example, a user may always choose the lowest priority in order to minimise cost. On the other hand, the user may always select the highest priority in order to minimise delay, irrespective of the cost incurred. The user may even randomly assign priorities to her traffic. However it is our contention that an intelligent priority selection strategy should be used, one based on the difference between the utility derived from playing the game with low delay and the cost incurred in doing so. In this paper we propose a utility-based algorithm that calculates the utility of the game to the user based on the roundtrip delay time experienced by packets. It also actively monitors the delay experienced by other traffic classes and based both on this and the per-byte or per-packet costs of the traffic classes, chooses which priority the user should assign to traffic. The delay is measured using an Exponentially Weighted Moving Average (EWMA) of the roundtrip time as shown in (1). The use of an EWMA helps smooth temporary fluctuations in delay, while still giving precedence to the most recent delay value. Di is the average after the ith sample; Mi is the value of the ith measurement and α is 0.94. Measurements are taken both from the application traffic generated and received on the current traffic class, as well as measurements from additional packets taken on each of the other available classes. These additional measurements incur an additional charge for the user with no corresponding
(1)
0.6
Utility
0.4 0.2 0 0
50
100
150
200
Roundtrip Delay
Fig. 2. User utility function for Counterstrike traffic
U max , U i ( Di ) = ( Dmax − Di )( U max ), Dth 0, CS i = U i ( Di ) − Ci
Di ≤ Dth Dth < Di ≤ Dmax Di > Dmax
(2)
(3)
To factor in the cost of the different traffic classes we apply (3). This metric is the utility minus the cost, also known as the Consumer Surplus (CS). As mentioned in section 2, utility and cost are considered directly comparable metrics. Periodically, the CS from each of the available traffic classes is calculated and the class returning the maximum value selected. Where cost is
4. Simulation setup The following experiments were carried out using the NS-2 simulator [24] with a Halflife (Counterstrike is a modification of the original Halflife engine) traffic generator module [25]. This traffic generator has been modified on both server and client side to facilitate the use of the priority selection algorithm discussed in section 3. During the simulation, a user communicates with a Counterstrike server located outside of the access network. The user's last mile link is well provisioned but the aggregate section of the regional access network may experience congestion on some of the downstream aggregate links. These links connect the Broadband Remote Access Server (BRAS) to the DSL Access Modules (DSLAMs) located in the various local exchanges. A 24:1 ratio is provided to the users sharing it, in line with the provisioning of many residential DSL networks. The background traffic on the aggregate link is generated using NS-2's exponential traffic sources. For simplicity, there are only two available priority levels, or traffic classes, Best Effort (BE) and High Priority (HP). The queuing mechanism is a straight-forward priority queue, traffic in the BE queue is only served when the HP queue is empty and both queues are FIFO tail-drop queues. The background traffic generated may range from light to heavy. No background traffic is transmitted at high priority. HP traffic incurs a cost of 0.4 units perpacket and BE traffic incurs a cost of 0.2 units per-packet. Since the maximum utility obtainable is 1.0 per-packet received, there is sufficient motivation for users to avail of either traffic class. The cost of HP traffic has been set at double the cost of BE traffic, to discourage the user from always simply availing of the higher priority class. The user may employ four different strategies. She may mark all her traffic as BE in order to minimize cost, she may mark all her traffic as HP to minimize delay or she may randomly assign a marking to her traffic. Alternatively, she may employ the CS priority selection strategy outlined in section 3, to mark her traffic. The disadvantage of the latter option is that the user must generate extra traffic, thereby incurring an additional cost, to monitor the delay on the traffic class she is not using. To do so, she transmits an echo-request every 500ms of similar size to the packets being generated by the Counterstrike application server. The strategy outcomes are compared for the three different scenarios of light, heavy and varied loads of background traffic.
TABLE I SIMULATION RESULTS Strategy Outcome (CS) CS Random Always LP Always HP
Background Traffic Light
Heavy
Varied
16224.3 15970.9 16637.6 13907.4
12102.52 1057.66 -6026.8 13884.1
15125.6 10362.9 4157.23 13893.46
5. Simulation results Table 1 shows the results obtained when the CS strategy is compared to the other available strategies. The best result for each network loading is highlighted in bold. As can be expected the static priority selection strategies perform well when traffic is either light or heavy for the entire duration of the simulation. They do not perform as well when the traffic varies, as it does in the final simulation. In all simulations, one of the static priority selection strategies is the worst strategy. When the background traffic varies, the CS strategy is the best available strategy despite the extra cost incurred in monitoring the other available traffic class. The CS strategy is always within 13% of the best strategy, regardless of the loading on the network. It always outperforms the worst strategy by at least 14% and in one case represents a nearly 150% improvement over the worst strategy. It is worth noting that, although the random priority selection strategy returns reasonable results in this scenario, it would have performed much worse had jitter been included as an input metric when deriving the utility. Roundtrip Delay Experienced by Gamers Low Priority 70
60
Roundtrip Delay in Milliseconds
applied by the network on a per-byte, rather than a perpacket basis, the mean packet size should be used in calculations.
50
40
30
20
10
0 2000
2100
2200
2300
2400 2500 2600 Simulation Time in Seconds
2700
2800
2900
Fig. 3. Roundtrip delay times for low priority strategy
an initial jump in the roundtrip delay time due to a sudden increase in background traffic load, the roundtrip delay experienced closely matches that of the high priority strategy.
Roundtrip Delay Experienced by Gamers Random 70
Roundtrip Delay in Milliseconds
60
50
6. Conclusions
40
30
20
10
0 2000
2100
2200
2300
2400 2500 2600 Simulation Time in Seconds
2700
2800
2900
Fig. 4. Roundtrip delay times for random strategy Roundtrip Delay Experienced by Gamers CS 70
Roundtrip Delay in Milliseconds
60
50
40
30
20
10
0 2000
2100
2200
2300
2400 2500 2600 Simulation Time in Seconds
2700
2800
2900
Fig. 5. Roundtrip delay times for CS strategy Roundtrip Delay Experienced by Gamers High Priority 70
Roundtrip Delay in Milliseconds
60
50
40
30
20
10
0 2000
2100
2200
2300
2400 2500 2600 Simulation Time in Seconds
2700
2800
2900
Fig. 6. Roundtrip delay times for high priority strategy In fig. 3-6, the roundtrip delays observed for the varied load simulation are shown, for each of the applicable strategies. As can be seen, for the CS strategy, apart from
In the future, it is probable that users will have a choice of priority levels or traffic classes to choose from. Users may have to actively indicate their desired service level by marking their application traffic. In such an environment a priority selection strategy may prove beneficial to users. In this paper, we have proposed a priority selection strategy based on Consumer Surplus (CS), a microeconomics term describing the positive difference between the amount the user is willing to pay (utility) and the actual cost. In a set of experiments we have compared the results from using this strategy with several other possible priority selection strategies. We have done so across a range of traffic loads, within a realistic representation of a residential broadband access network. We have shown results for users involved in a multiplayer Counterstrike game. Online multiplayer games have demanding QoS requirements and are increasing in popularity. It was found that the CS algorithm proposed is a viable strategy for choosing the priority level in a future QoSaware network. For all simulations it was within 14% of the best strategy and when the network loading was varied during the course of the simulation, it proved to be the best strategy. The strategy is not without its weaknesses. Users must incur the cost of monitoring the other available traffic classes. Though the frequency of the measurements may be reduced, there is a consequent loss of accuracy in doing so. Furthermore, although the strategy proved useful for networks with varied traffic loads, the user is using previous network state information to influence her current traffic class choices. This is not guaranteed to be accurate and indeed, it is possible to envision scenarios, however unlikely, where this would result in poorer application performance. Excepting the case of networks where information regarding the current network state is presented to the user, the user has little choice but to rely on this information. The strategy would be applicable not only to heterogeneous IP networks with multiple traffic classes but also one where networks are logically separated, with pricing alone used to prevent congestion as outlined in [12].
7. Future work The research presented here has a number of interesting avenues open to further exploration. Although we have considered a packet-level QoS solution for heterogeneous networks, it has not yet been tested in any wireless networking scenarios. In future work we hope to investigate the applicability of the strategy to wireless networks and for a range of other applications such as interactive voice and video. We will also seek to explore the effect of increasing the number of traffic classes available to the user, as well as that of running multiple services over the last-mile link to the customer’s premises. Anther issue not considered in this work is the effect of relative delay. Users may be willing to tolerate a high absolute delay, provided the delay difference between users is low. Work in [24] suggests that users are more concerned with absolute delay when playing games but this may be due to a lack of feedback regarding the absolute delay times of other users.
8. References [1] R. Braden, D. Clark and S. Shenker, “Integrated services in the Internet architecture”, IETF RFC 1633, June 1994. [2] S. Blake et al., “An architecture for differentiated services”, IETF RFC 2475, Dec. 1998. [3] S. Zander, T. T. T. Nguyen and G. Armitage, “Automated traffic classification and application identification using machine learning”, In Proceedings of IEEE 30th Conference on Local Computer Networks (LCN’05), Sydney, Australia, 1517 November 2005. [4] Wikipedia, “Counter-Strike”, http://en.wikipedia.org/wiki/Counter-Strike, last accessed June 1, 2006. [5] C. Courcoubetis, G. D. Stamoulis, C. Manolakis, and F. P. Kelly, “An intelligent agent for optimizing QoSfor-money in priced ABR connections,” Telecommunications Systems, Special Issue on Network Economics, to appear. [6] S.K. Joyce, “Traffic on the Internet – a study of Internet games”, Network Analysis Times, vol. 2(1), pp. 6-8, April 2001. [7] C. Schaefer, T. Enderes, H. Ritter and M. Zitterbart, “Subjective quality assessment for multiplayer realtime games”, In Proceedings of the 1st Workshop on
Network and System Support for Games (NetGames’02), pp. 74-78, Braunschweig, Germany, Apr. 2002. [8] T. Henderson and S. Bhatti, “Networked games: A QoS-sensitive application for QoS-insensitive users?” In Proceedings of ACM SIGCOMM 2003 Workshops, p. 141-147, Aug. 2003. [9] C. Fraleigh, F. Tobagi and C. Diot, “Provisioning IP backbone networks to support latency sensitive traffic”, In Proceedings of IEEE INFOCOM 2003, vol.1, Mar. 2003, pp. 375-385 [10] H. Varian, “Intermediate Microeconomics: A Modern Approach, 6th Edition”, WW Norton Publishing, 2003. [11] M. May, J. Bolot, C. Diot and A. Jean-Marie, “1-bit schemes for service discrimination in the Internet. Analysis and evaluation”, Technical Report 3238, INRIA, 1997. [12] A. Odlyzko, “Paris metro pricing: The minimalist differentiated services solution”, In Proceedings IEE/IFIP 7th International Workshop on Quality of Service (IWQoS’99), pp. 159-161, June 1999. [13] B. Carrig, D. Denieffe and J. Murphy, “Delivering Quality of Service for gaming applications in best effort networks”, In Proceedings of the Thirteenth International Conference on Telecommunications (ICT’06), Madeira Island, Portugal, May 2006. [14] B. Tuffin, “Charging the Internet without reservation: An overview and a bibliography of approaches”, Journal of Information Science and Engineering, vol. 19, pp. 765-786, Mar. 2003. [15] Y. Hayel, D. Ros and B. Tuffin, “Less-than-besteffort: Pricing and scheduling”, In Proceedings of IEEE INFOCOM 2004, vol. 1, Mar. 2004, pp. 66-75. [16] M. Mandjes, “Pricing strategies under heterogeneous service requirements”, In Proceedings of IEEE INFOCOM 2003, vol. 2, Mar. 2003, pp. 1210-1220. [17] T. Jehaes et al., “Access network delay in networked games”, In Proceedings of the 2nd Workshop on Network and System Support for Games (NetGames’03), pp. 63-71, Redwood City, California, Apr. 2003. [18] G. Ballintijn, M. van Steen and A.S. Tanenbaum, “Characterizing Internet performance to support
wide-area application development”, ACM SIGOPS Operating Systems Review, 34(4), pp. 41-47, Oct. 2000.
[23] T. Lang, P. Branch and G. Armitage, “A Synthetic Traffic Model for Quake3”, In Proceedings of ACM SIGCHI ACE2004, Singapore, June 2004.
[19] G. Armitage, “An experimental estimation of latency sensitivity in multiplayer Quake3”, In Proceedings 11th IEEE International Conference on Networks (ICON’03), Sydney, Australia, Sept. 2003, pp. 137141.
[24] T. Henderson, “The effects of relative delay on networked games”, Ph.D. thesis, University of London, London, UK, April 2003. Available online at http://www.cs.dartmouth.edu/~tristan/pubs/thesis.pdf
[20] Slashdot.org, “How fast too slow? A study of Quake pings”, May 24, 2001. http://slashdot.org/articles/01/05/24/2044233.shtml [21] S. Zander and G. Armitage, “Empirically measuring the QoS sensitivity of interactive online game players”, In Proceedings of Australian Telecommunications Networks and Applications Conference (ATNAC2004), Sydney, Australia, December 2004. [22] NS v2 Simulator. http://www.isi.edu/nsnam/ns
[25] P. Quax et al., “Objective and subjective evaluation of the influence of small amounts of delay and jitter on a recent first person shooter game”, In Proceedings of ACM SIGCOMM 2004 Workshops, pp.152-156, Aug. 2004. [26] W. Feng et al., “A traffic characterization of popular on-line games”, In IEEE/ACM Transactions on Networking, vol. 13, no. 3, pp.488-500, June 2005. [27] J. Farber, “Network games traffic modelling”, In Proceedings of the 1st Workshop on Network and System Support for Games (NetGames’02), pp. 5357, Braunschweig, Germany, Apr. 2002.
