cloud gaming: a green solution to massive multiplayer ... - IEEE Xplore

CHUAH_LAYOUT_Layout 8/13/14 2:41 PM Page 78

GREEN MEDIA

CLOUD GAMING: A GREEN SOLUTION TO MASSIVE MULTIPLAYER ONLINE GAMES SEONG-PING CHUAH, CHAU YUEN, AND NGAI-MAN CHEUNG

ABSTRACT

The authors are with Singapore University of Technology and Design. This research is supported by the Multi-plAtform Game Innovation Centre (MAGIC), funded by the Singapore National Research Foundation under its IDM Futures Funding Initiative and administered by the Interactive & Digital Media Programme Office, Media Development Authority. 1

community.callofduty. com/community/call_of_ duty/english/ghosts/blog/ 2013/10/23/call-of-dutyghosts-pc-systemrequirements 2

www.nvidia.com/object/ cloud-gaming.html

78

Advanced video gaming is a computationally intensive application. Sophisticated graphics renderings are employed in computer games to produce realistic scenes and smooth actions. As a result, video gaming often requires powerful hardware that is beyond the capability of many mobile devices or even personal computers. Meanwhile, playing a high-quality game while on the move is highly desirable with the growing popularity of high-speed mobile and broadband Internet, and mobile devices such as smartphones and tablets. Instead of equipping mobile devices with powerful but battery-hungry computation engines, cloud gaming, which utilizes cloud computing for gaming, offers an emerging green solution to bring the high-quality immersive gaming experience to thin or mobile clients. Cloud gaming leverages communication infrastructures to shift heavy computation to cloud servers. In this article, we provide an overview of cloud gaming from a green media perspective (in addition to the conventional energy perspective). We argue that cloud gaming can lead to less software maintenance, more economical scaling, and longer service life spans of hardware equipment. We also briefly present a novel scheme, layered coding, which leverages the increasing graphics processing capability of a mobile client to reduce the bit rate of game streaming. We then discuss green designs of major cloud gaming subsystems: a cloud data center, graphics rendering, video compression, and network delivery. We review existing services and a testbed for cloud gaming. We also identify potential research challenges of cloud gaming in achieving green media for the future.

INTRODUCTION Video gaming is one of the fastest growing business in multimedia entertainment industry. Fueled by the large-scale adoption of smart mobile devices, mobile games such as Angry Birds and Candy Crush have surprised many by hitting a huge number of players in a short period of time. Despite tremendous growth of capability in mobile devices, gaps remain between what mobile devices can support and requirements of great video games. High-end video

1536-1284/14/$25.00 © 2014 IEEE

games often require advanced computer graphics and enormous computational power to render realistic and interactive gaming scenes. Intensive computations for game rendering drain the batteries of mobile devices quickly. Battery capacity has been lagging behind the growth of computation power in mobile devices. As a result, mobile devices are crippled by the limited power supply in running computationally intensive applications. In addition, video games require substantial storage in installation, thanks to their high-fidelity textured graphics. For example, a popular car racing game on smartphone Asphalt 8 requires 901 Mbytes of storage, which is about 6 percent of storage on a 16Gbyte smartphone. Not to mention the recent release of the high-end video game Call of Duty: Ghost, which requires minimum specifications of a dual core CPU at 2.66 GHz, a graphics card with at least 1 Gbyte RAM, 6 Gbytes of system RAM, and 40 Gbytes of storage space, which is only possible if the game is installed on a PC or game machine.1 Such specifications could not be supported by even the most powerful mobile device on the market. Recently, cloud computing has attracted substantial interest in the research community [1] and is widely deployed in industry.2 Cloud computing provides resources of powerful servers to thin clients connected via the Internet by leveraging on the broadband Internet, or high-speed LTE or 4G wireless networks. Cloud computing can be categorized into three major services: storage as a service, processing as a service, and software as a service. Storage as a service, such as Dropbox, allocates certain storage space to users for ubiquitous and pervasive access from any device. Processing as a service, such as Amazon EC2, provides users with high-performance computing resources for computation-intensive tasks. Software as a service, such as Google Drive and Microsoft Office 365, allows users to use software functions run on remote servers. An analysis of energy consumption of data storage, processing, and transmission in cloud computing has been widely studied in the literature. It was shown in [2] that cloud computing provides a green solution to computational needs. In cloud gaming, game software is hosted in cloud servers. Users’ controls and button presses at the clients are transmitted to the cloud

IEEE Wireless Communications • August 2014


servers. Game logics are executed at the cloud servers, and game images are rendered. The rendered game images are then streamed back to the client. Given the resource-intensive nature of video games, cloud gaming offers an attractive solution for a pervasive and ubiquitous gaming experience for mobile devices. Cloud gaming offers overall green efficiency in the following aspects.

Software Management — Installing software on individual local machines require regular updates for patches and fixes. Achieving compatibility with different operating systems and platforms requires significant effort and resources. Software maintenance for a broad range of platforms incurs substantial operating cost for software vendors. When new versions of a game are marketed, updates and support for older versions are often terminated gradually. Cloud gaming has the advantages of making new releases available to client immediately, and solving compatibility and update issues. As game vendors only need to maintain the game software at cloud servers, software management becomes more cost effective and requires less resources. Hardware Maintenance — New video games with better graphics demand higher hardware capability. Hardware often needs to be upgraded to fulfill the requirements in traditional game installations at local machines (game consoles, PCs, etc.). Hardware may be abandoned because it cannot meet the requirements of new games, even though they function fine. As a result, unnecessary electronic wastes are produced. A recent report3 revealed that in the United States alone, annual e-waste disposal tops 372.2 million units, of which computers and cell phones make up 93 percent. In cloud gaming, users are not required to upgrade and maintain powerful hardware, since high-quality video games are rendered at the cloud servers. Besides, video games often require large storage space for installations and push up the demand for larger storage disk in consumer electronic devices. Installation spaces are highly duplicated in gaming on local private machines. In cloud gaming, games are rendered at cloud servers with abundant and scalable resources. Importantly, cloud gaming centrally manages a pool of storage and computing resources rather than each client being constrained by their resources. Economies of scale and statistical multiplexing of client usage ensure overall efficiency in hardware utilization. Besides, power consumption of user devices is reduced as power consuming computations are carried out in cloud servers, which are often power optimized. Deployment Cost — Cloud gaming eliminates the cost for sale and distribution of video games by delivering the games to massive numbers of players directly. Besides, pay-per-service saves clients from the large upfront costs of owning hardware and software in which they are likely to lose interest sometime later. For example, if the game Call of Duty is shifted to the cloud and streamed to a user at 720p and 30 fps (i.e., about 3 Mb/s), cloud gaming will reduce the overall


volume of data transfer if the game is played less than 3.7 h (which is equivalent to 40 Gbytes of downloading!). Hence, this enables a user to have a quick trial of the game without the need for heavy downloading. This business model also provides better protection of copyrights and avoids game piracy. In summary, cloud gaming offers green solutions in terms of better manageability of game software, less storage and equipment wastage due to hardware incompatibility, and more costeffective deployment to massive numbers of client. In this article, we present an overview of cloud gaming in green aspects by first presenting green designs of technical modules in cloud gaming. We concentrate on the cloud data center, game graphics rendering, video coding and compression, and network delivery. Then we introduce some existing commercial services and an open source testbed of cloud gaming. We also briefly introduce our novel cloud gaming framework, and evaluate its performance in relation to the conventional framework. Finally, we explore some potential directions for cloud gaming in achieving green media through various designs and optimization frameworks.

The data center is the processing hub of cloud computing where a pool of computing and storage resources are managed and mobilized to serve clients. Offloading these computations to the cloud offers better utilization of and power in the cloud data center than on individual machines.

GREEN DESIGN OF CLOUD GAMING In cloud gaming, mobile clients are often connected to the Internet via high-speed wireless networks such as Long Term Evolution (LTE), fourth generation (4G), and WiFi, while cloud servers and home clients are linked to core networks via fixed line broadband networks. The framework of cloud gaming consists of a graphics rendering pipeline, video compression, and network delivery modules, as shown in Fig. 1. In this section, we present a brief overview of energy-efficient designs in each module of the cloud gaming platform.

GREEN CLOUD DATA CENTER The data center is the processing hub of cloud computing where a pool of computing and storage resources are managed and mobilized to serve clients. Offloading these computations to the cloud offers better utilization of hardware and power in the cloud data center than on individual machines, thanks to the economy of scale where the cost of a service/product is reduced as the production is scaled up [3]. In the cloud gaming applications, game graphics rendering and video coding of the rendered images are performed at such green data centers. To complete the computation task in a cloud data center, virtual machine (VM) technology has been widely deployed, thanks to its flexibility, reliability, and ease of management. VM technology allows a physical computer to run multiple operating systems (OSs) with strong isolation among the OSs. VMs share the physical hardware that allocates resources among the VMs. Computing resources can therefore be scaled by running more or fewer physical machines according to the client’s demands. Energy-efficient resource allocation of computing resources in a data center has been widely studied in the literature. For example, aiming to minimize the power consumption of cloud a

3 www.theguardian.com/ news/datablog/2012/mar/ 16/infographics-datavisualisation-history

79


Realistic and smooth game graphics are essential for a good gaming experience. However, such rendering is computationally intensive. As graphics rendering involves the processing of a large block of parallel data, graphics processing units are often deployed instead of conventional central processing units.

Cloud servers Game logic

Game actions

Command emulator

Thin clients Player commands

www.nvidia.com/object/ nvidia-grid.html 5

shield.nvidia.com

80

Player interaction

Video bitstream

Video decoder

Rendering commands Networks

Graphics rendering Rendered images Video encoder

Encoded video

Real-time streaming

Video bitstream

Figure 1. Cloud gaming framework.

data center while ensuring performance from the user perspective, Liu et al. [2] proposed the GreenCloud architecture, which allows online monitoring, live migration of VMs, and VM placement optimization. A similar effort in this pursuit, Beloglazov et al. [4], investigated a heuristic method for energy-efficient management of a data center for cloud computing. Resource allocation and scheduling policies were formulated to meet the requirements of quality of service and power usage characteristics.

ENERGY-AWARE GRAPHICS RENDERING

4

Player commands

Realistic and smooth game graphics are essential for a good gaming experience. However, such rendering is computationally intensive. As graphics rendering involves the processing of a large block of parallel data, graphics processing units (GPUs) are often deployed instead of conventional central processing units (CPUs). In terms of rendering hardware for cloud gaming, the power consumption of GPUs can be controlled via dynamic voltage-frequency scaling (DVFS) [5]. Power consumption of gate switching in complementary metal oxide semiconductor (CMOS) integrated circuits are proportionate to the cube of clocking frequency. Thus, power consumption can be substantially reduced if the operating frequency is clocked slower. When the DVFS scheme is incorporated with accurate prediction of rendering load, the energy efficiency of GPUs can be improved by adapting voltagefrequency, as demonstrated in [6]. At low rendering load, the voltage-frequency of GPUs can be tuned more slowly, thus consuming less power without affecting the rendering tasks. In terms of game rendering algorithms, each stage of the rendering pipeline incurs some computation complexity (thus computing power). The energy models of the rendering pipeline have been characterized in [7], which demonstrated that processing of geometry, fragments, and game logic consume the most power in the pipeline. In general, rendering quality and complexity (thus energy consumption) trade off various aspects of graphics rendering. Resolution: Resolution includes the frame size and frame rate of a game video. In rendering a pipeline, the color value of each pixel is computed using a sophisticated rendering algo-

rithm. Thus, rendering complexity increases substantially with the increase of game display resolution. Resolution can be adapted according to the client’s device screen and capability, which we discuss more thoroughly below. Level of detail: Number of polygon (vertex data) defines the level of detail of a 3D game object. More polygons provide better details on a 3D model surface at the cost of more energy consumption in geometry processing. Lighting and shading: Shading and lighting describe the depth and color illumination of a 3D scene. Photo-realistic rendering effects require complicated algorithms to simulate light paths. Unlike 3D animation in films, video games are often action-intensive and do not require full simulation of light rays. Thus, a reasonable balance could be explored. Texture/normal/bump map: A texture map adds surface details and pattern to a 3D model, while normal and bump maps simulate rough surfaces. Mappings provide realistic effects on a 3D surface, but require more computations. Power consumption and the computation bottleneck in rendering a pipeline depend largely on the game animations, which vary in the aspects listed above. By optimizing the rendering pipeline according to the gaming animation, energy efficiency can be achieved via computationally efficient rendering algorithms in computer graphics. Along this direction, Nvidia has introduced GRID 4 and SHIELD, 5 which allow more efficient rendering of high-quality graphics in remote PCs powered by powerful GPUs.

ENERGY-EFFICIENT VIDEO COMPRESSION A graphic rendering pipeline renders game animation into a sequence of images that is compressed into a video bitstream. In video compression, there is a trade-off in the optimization of rate and distortion. To compress an image sequence, redundancies are removed via searches of motion vectors and coding mode. In general, better rate distortion performance can be attained when computation resources (power consumption) is unconstrained such that the best motion vectors and coding mode decisions can be obtained via thorough searches. Searching for these coding decisions is most computationally demanding in video cod-



ing. Limiting the search space of the motion vectors and mode decisions can scale the power consumption and rate distortion of video coding, as shown in the literature [8]. By investigating the complexities of each module in the coding pipeline, Su et al. [8] demonstrated video coding schemes with complexity scalability in video coding design. Similar to the graphics rendering, energy efficiency of video coding can be achieved by integrating the control of algorithmic complexity with the voltage-frequency of CPUs via DVFS. For instance, He et al. [9] investigated joint adaptation of video coding and wireless transmission under a limited power budget. High compression efficiency results in lower bit rates, which can ease the stringent requirements on throughput and latency in network delivery, as we discuss later.

GREEN COMMUNICATIONS AND NETWORKING Compressed bitstreams from the video encoder are delivered to clients via the Internet. Game videos require high network throughput and low latency to ensure smooth playout. These stringent demands are often power consuming due to the energy-delay trade-off in communication networks. Energy efficiency has been a major focus in communications research. Broad surveys on energy consumption of communication networks were presented in [10]. Wireline access networks such as digital subscriber line (DSL) consume power at a fixed rate, disregarding the access data rate. A straightforward solution to energy efficiency is to increase the power efficiency of their components, such as electronics circuits, power amplifiers, and power converters. However, these components are rather mature where room for improvement is limited. More sophisticated methods include intelligent management of nodes via software-defined networking (SDN) and dynamic topology optimization [10]. SDN allows network administrator to manage network services through the abstraction of lower-layer functionality. SDN can potentially simplify the tasks of connecting applications and network appliances in the cloud, social networking, and mobile devices in multiplayer online games. 6 Game providers can leverage on SDN to adapt to the dynamic changing of a gamer’s demands without having to reconfigure or replace the network’s hardware components. By reorganizing the network topology during off-peak hours via dynamic topology optimization, traffic demands can be supported by fewer operating nodes, thus reducing the overall energy consumption. However, given the stringent latency constraint of cloud gaming, optimizing the networks is challenging since fewer operating nodes means less network capacity, longer routing path, and queue delay. Nevertheless, these challenges can be overcome via multipath transmissions in heterogeneous networks, as demonstrated in [11]. In wireless access networks such as LTE, the energy efficiency of wireless networks can further be improved via transmission power. The transmission power of wireless access networks depends on the coverage distance and data rate, as stated in Shannon’s channel capacity.


A smaller cell size improves power efficiency by reducing the transmission power substantially, as wireless channel loss is lower in a shorter distance. By deploying relay or femtocell/hotspot, high-data-rate connectivity can be ensured at lower transmission power. However, the additional power consumption of femtocell nodes should be factored in. In addition, there are several emerging communication techniques, such as cognitive radio networks and massive multiple-input multiple-output (MIMO) antenna systems. In cognitive radio networks, available wireless spectrum can be dynamically exploited for higher channel capacity and better energy efficiency in wireless transmissions. Similarly, MIMO antenna systems increase the channel capacity of a wireless link by exploiting the spatial multiplexing gain. It has been shown that more antennas in a massive MIMO system are more energy-efficient than single-antenna systems. Overall, joint optimization of a cloud data center, graphic rendering, video compression, and communications over wired and wireless networks is necessary to meet the stringent requirements of cloud gaming while ensuring energy efficiency. For example, the authors in [12] explored a green reconfigurable router that controls power dissipation with power-aware routing, rate-adaptive processing (via DVFS), and power-efficient architecture. Each terminal or base station in the wired and wireless networks can be modeled as a green reconfigurable router in the optimization.

In wireless access networks such as LTE, the energy efficiency of wireless networks can further be improved via transmission power. The transmission power of wireless access networks depends on the coverage distance and data rate, as stated in Shannon’s channel capacity.

CLOUD GAMING PLATFORM In this section, we present some existing platforms of cloud gaming, both proprietary and open source.

COMMERCIAL SERVICES Cloud gaming is an emerging application with promising potential. Currently, several commercial services and research testbeds demonstrating the functionality of cloud gaming are being developed. Commercial services include StreamMyGame,7 OnLive Game System,8 and Gaikai’s streaming service.9 StreamMyGame is a software solution that streams Windows-based games to remote Windows and Linux devices on which a user should install server and player applications. Onlive Game System provides games to various devices, including PCs, tablets, smartphones, TV set-top boxes, and other devices. Similar to StreamMyGame, a user needs to install a client terminal application to connect to its cloud data center. Gaikai’s streaming service can be embedded into websites (including social media sites) and devices (e.g., mobile devices and smart TVs), so users are not required to install any software. The market potential of such websiteembedded cloud gaming is demonstrated by the acquisition of Gaikai by Sony in 2012. For game video at 720p (no frame rate has been specified though), StreamMyGame requires bandwidth of 6–8 Mb/s, while Onlive recommends at least 5 Mb/s. On the server side, Nvidia has introduced the GRID cloud as an advanced game server

6 www.ibm.com/developerworks/community/blog s/ibmsyssw/entry/are_ga ming_companies_adopting_software_defined_as _part_of_their_strategies?lang=en. 7 www.streammygame. com 8

games.onlive.com

9

www.gaikai.com

81


The open source alternative to the commercial platforms is GamingAnywhere, which aims to be an open platform for real-time streaming application for multimedia cloud deployment. GamingAnywhere is the first open source cloud gaming platform for research community in this pursuit.

10

kanaka.github.io/ noVNC/noVNC/vnc.html

82

that is able to support 48 HD-quality game streams. More recently, Nvidia has introduced SHIELD, a mobile gaming device that connects to a desktop PC or the GRID Cloud via 802.11n 2 × 2 MIMO. Users can still play high-quality games, which are rendered at the GPU-equipped desktop PC and streamed to SHIELD while they are away from the PC. Due to the proprietary nature of the services, no further technical details on these services are available. However, evaluations of some of these services were reported in [1].

OPEN SOURCE TESTBED: GAMINGANYWHERE The open source alternative to the commercial platforms is GamingAnywhere [1], which aims to be an open platform for a real-time streaming application for multimedia cloud deployment. GamingAnywhere is the first open source cloud gaming platform for the research community in this pursuit. GamingAnywhere has been designed with extensibility, portability, configurability, and openness as the objectives. It supports major platforms such as Windows, Linux, and OS X, where the data flow carries the compressed multimedia streams of video games from servers to clients, while the control flow carries the user’s actions on the opposite direction. A GamingAnywhere server consists of games, a multimedia coder, an input emulator, and an RTSP server. Any game that can be installed and run on a PC can be played on the GamingAnywhere platform. It serves the goals of cloud gaming when the games are of high-end with demanding GPU operations. The current implementation of GamingAnywhere supports encoding of one-encoder-for-all and one-encoder-per-client modes. In one-encoderfor-all mode, game images are encoded by an encoder regardless of the number of observers. This mode has the benefit of better efficiency as CPU usage remains the same when there are many observers. On the other hand, oneencoder-per-client allocates an encoder for each client or observer. The advantage of this mode is the flexibility of using different encoding configurations, such as spatial, temporal, quality resolutions, bit rate, and so on. Similar to a commercial cloud gaming service, GamingAnywhere requires a client to install delicate client software to render the game at the client side. A GamingAnywhere client consists of an RTSP client, multimedia decoder, and SDL for RTP stream handling, video bitstream decoding, and input event capturing. In performance evaluations [1], GamingAnywhere is found to be more responsive (less delay), less bandwidth consuming, and better video quality compared to some existing commercial services.

types of OSs, such as Microsoft Windows, X11 in UNIX, and so on. Pixel format and encoding in RFB are determined from the negotiation between the server and client. The server must encode in the way the client is able to decode. If the client is able to cope with several formats and encodings, it should choose one that is easier for the server to encode. Among the encoding schemes include raw encoding with simply no compression; copy rectangle encoding, which copies from the previous frame; and RRE/ CoRRE/Hextile encoding, based on run length encoding. Pixel formats include 8-bit color map and 16-bit/24-bit true color. In our cloud gaming deployment, VNC servers are hosted on the cloud that hosts instances of game play. The game screens on the cloud servers are rendered to clients as a remote window. To serve VNC clients on web browsers that run the Websocket protocol, Websocket proxies on the server side are normally necessary to bridge the communications. On the client side, our platform deploys noVNC,10 which is a browser-based VNC client implemented using HTML5 Canvass and Websockets. As most browsers are HTML5-enabled, using a VNC client as a gaming terminal in a client’s browser enables platform-independent gaming. Our VNC-based cloud gaming promises the advantage of platform-independent gaming. As games are delivered via browser, game providers and players are free from the hassle of hardware/software compatibility issues and maintenance. In contrast, GamingAnywhere requires installation of delicate software on clients, thus requiring maintenance effort to ensure that it works on the supported OS or devices. However, as RFB only removes spatial and entropy redundancies, the compression efficiency is much lower than the motion-compensated transform coding in GamingAnywhere. Thus, VNC-based cloud gaming is more suitable for less motionintensive games such as strategy, maze, and such games, while motion-intensive games such as first-person shooter games and racing games fit better on GamingAnywhere.

NOVEL ARCHITECTURE FOR MOBILE CLOUD GAMING In this section, we give an overview of a new architecture for mobile cloud gaming. With the increasingly availability of rendering in mobile devices, we discuss a framework to leverage the rendering of mobile devices to reduce the transmission bandwidth of game images. Detailed discussion of the proposed framework can be found in [13].

VNC-BASED CLOUD GAMING PLATFORM

SYSTEM FRAMEWORK

At the Singapore University of Technology and Design (SUTD), we aim to develop a platformindependent gaming architecture. We set up a cloud gaming platform based on virtual network computing (VNC). VNC is a graphical remote desktop sharing system that adopts the remote frame buffer (RFB) protocol. As the RFB protocol works at the frame buffer level, it enables compatibility of remote desktop access to all

Most existing cloud gaming platforms render graphic images and encode them into a video bitstream at the cloud servers, and deliver the video bitstream to the mobile client. Increasingly, however, modern mobile devices equipped with more powerful mobile GPUs have some rendering capability. In the layered coding scheme, we propose that mobile clients can render low-quality game images or the base layer,



Rendering commands

High quality graphic renderer

Base layer graphic renderer Game logic

Client’s actions

IBL IHQ IEL

Image / video decoder

Image / video decoder

IBL

^

I

EL ^

I

Enhancement layer information, T(IEL) Game actions

Cloud servers

HQ

High quality graphics display Mobile client

Figure 2. Mobile cloud gaming system with layered coding. IBL. Instead of sending high-quality game images, I HQ , cloud servers can transmit enhancement layer images, IEL, to the clients to improve the quality of the base layer. The proposed cloud gaming framework is illustrated in Fig. 2. IEL are generated as the pixel-wise difference between IHQ and IBL. IEL are encoded using H.264/AVC transform coding, denoted as T(.). The transform-coded IEL, T(IEL), constitutes the enhancement layer information sent from the cloud server to the mobile device. Upon receipt of T(I EL ), the mobile device performs transform decoding T –1 (.) to recover the enhancement image, and adds it to its own IBL to produce ^ IHQ (reconstruction of IHQ): Server computation:T(IEL) = T(IHQ – IBL) (1) Client reconstruction: ^ IHQ = T–1(T(IEL)) + IBL Note that as T(.) is a lossy coding algorithm, IHQ and ^ IHQ are not exactly the same. The main advantage of the layered coding is the lower bit rate of gaming streaming. Instead of sending IHQ, which contains substantial information content, layered coding transmits I EL , which is the difference between I HQ and I BL . Thus, T(I EL ) has a lower bit rate than T(I HQ ). Bit rate reduction of game streaming is crucial for low latency (and hence more responsive game play) and energy efficiency in network delivery, which can be more power consuming than CPU/GPU processing. To minimize the bit rate of T(I EL ), we optimize the rendering pipeline of IBL such that the information content of IEL is minimal while the rendering complexity of IBL is within the capacity of the mobile device. We briefly discuss the design of the rendering pipeline for IBL next.

DESIGN OF THE BASE LAYER PIPELINE In a graphics rendering pipeline, a sequence of rendering steps introduce various bits of visual information to the rendered image. Different rendering steps require different amounts of computation depending on the rendering parameters. In the design of a rendering pipeline for IBL, we identify the one that renders substantial visual information, but incurs few rendering computations. We employ this pipeline design to render IBL such that IEL are more compressible. In I BL rendering, we adjust the rendering pipeline as explained earlier. Major alterations


of the graphics rendering pipeline of IBL are as follows.

Polygon/Vertex — Low polygon models of game objects are adopted to provide coarse geometric structures of the game world while significantly reducing the geometry processing in the rendering pipeline. The low polygon models may contain only 20 percent of the full polygon models.

Most of the existing cloud gaming platforms render graphic images and encode them into video bitstream at the cloud servers, and deliver the video bitstream to the mobile client. Increasingly however, modern mobile devices equipped with more powerful mobile GPU have some rendering capability.

Lighting and Shading — Phong shading and lighting is an illumination model widely used in most graphics card. Phong lighting consists of three components: ambient (uniform color), diffuse (smooth shading), and specular (sparse bright reflection). The specular component, which is computation-intensive but contributes little information, is omitted from the IBL pipeline. Texture/Normal/Bump Mapping — Texture can be compressed efficiently as an image by existing image/video codecs. Thus, texture mapping is omitted from the IBL pipeline. However, normal or bump mapping are included because they are required in the diffuse shading of Phong lighting.

PERFORMANCE EVALUATION To illustrate the performance of the discussed mobile cloud gaming architecture, we use Blender,11 a popular graphic rendering software for game content creation, to render a game-like animation, Wormhole at 720p 30 fps. Sample images of IHQ and IBL are shown in Figs. 3a and 3b, respectively. It can be seen that I BL is the coarse version of IHQ. IEL are obtained as in Eq. 1 and encoded using H.264/AVC. Figure 3c shows the rate-distortion (RD) performance of layered coding and direct coding. Direct coding, which encodes and streams I HQ as a video to clients, is used in the cloud gaming industry and some cloud gaming platforms such as GamingAnywhere. The top left of the RD curves shows that the layered coding achieves lower bit rate at video quality similar to that of direct coding.

FUTURE DIRECTIONS AND CHALLENGES The existing commercial services and research testbed have showcased the advantage of cloud video gaming where high-end video games are rendered at clouds and delivered to thin clients for remote gameplay. While the desired features

11

www.blender.org

83


PSNR (dB)

50 65.6% lower bitrate 40

30

(a)

(b)

Direct coding Layered coding 0

0.05

0.1 0.15 Bit/pixel

0.2

(c)

Figure 3. Experiments using a game-like animation sample, Wormhole: a) high-quality image IHQ; b) base layer image IBL; c) ratedistortion performance. have been demonstrated, we identify some future directions in green designs that can further improve energy efficiency of cloud gaming.

PLATFORM-INDEPENDENCE GAMING Most of the services and the testbed require installation of client software or browser plugin, which are OS platform or browser dependent. This requirement results in a lack of portability of the cloud gaming deployment, since new software/plugins are developed whenever a new OS and/or browser emerges. As an OS/browser evolves over time, maintaining the compatibility of these client softwares/plugins with the OS platforms requires additional effort. Besides, support for less popular OSs and plugins could be neglected for cost saving by service providers. A platform-independent solution is desirable for simplicity and low cost of maintenance for the cloud gaming provider. From the client’s perspective, the lifetime of equipment can be extended by enjoying cloud service while eliminating the need for frequent software or hardware upgrading. Thanks to the recent advance of web-based technology, browsers have emerged as an important platform for widely popular web applications. Web applications are getting more popular in providing software services to thin clients independent of OS platform. HTML5, a markup language which provides cross-platform support of multimedia and graphics without the need to install plugins, was recently finalized. The introduction of HTML5 is a promising step in realizing cross-platform design of cloud gaming. With various devices (PCs, TVs, laptops, tablets, and smartphones) equipped with web HTML5-ready browsers, high-end 3D video gaming can be delivered to thin clients via this cross-platform web technology. More important, HTML5 was designed considering operating on low-power mobile devices. WebGL is another web technology that provides an application programming interface (API) for 3D graphics over browsers based on HTML5 canvas elements. Access to hardware acceleration for graphics processing in an HTML5 browser allows web games to be rendered at previously unseen graphics quality. HTML5 with WebGL is promising for a truly platform-independent cloud gaming solution.

84

CROSS-LAYER RENDERING-CODING-DELIVERY OPTIMIZATION Three major technical modules of cloud gaming — graphics rendering, video compression, and communication networking — have been presented separately earlier. Cross-layer design of these modules can be applied to improve energy efficiency. Cross-layer optimization for conventional live video streaming has been well studied in the literature [14]. A network-aware video encoder can adapt the video bitstream based on network statistics to ensure quality of experience in gaming. For example, video bit rate is reduced with graceful degradation for smooth game video playout when the network is congested. Such a feature is extremely important for gaming as it is highly interactive. Having to wait for buffering significantly reduces the joy of gaming. On the other hand, content-aware network delivery algorithms could prioritize the video packets such that important and urgent video packets are delivered on time and reliably. Joint coding and delivery optimization avoids unnecessary waste of computational and communications energy due to poor coordination of these two technical modules. For cloud gaming, game graphics rendering is upstream of video encoding. Cross-layer design of joint rendering-coding optimization can be explored for better efficiency of computation. Specifically, rendering could adapt according to video compression. For example, less noticeable but computationally expensive rendering effects such as global illumination can be omitted when a video bitstream is to be substantially compressed. Global illumination simulates the light rays reflected by other surfaces to produce photo-realistic effects. On the other hand, motion physics in a 3D animation could possibly serve as accurate predictors in speeding up the motion search in video coding. Such cross-layer designs would potentially improve the energy efficiency of computation substantially. Different game types have different tolerable delay thresholds for satisfactory gaming experience. For example, first person shooter (FPS) and racing games have a delay tolerance of 100 ms, while real-time strategy games can tolerate up to 1s delay [15].



Depending on the game type, video coding and network delivery can allocate their resources such that all game videos are delivered to their clients before their respective tolerated delays. Exploiting different delay thresholds allows game-aware coding and delivery to improve energy efficiency from the energy-delay trade-off in computing and communications. In general, a cross-layer framework for joint rendering-coding-delivery optimization can be explored to improve the energy efficiency. Figure 4 shows the cross-layer framework of joint rendering-coding-delivery optimization. In such a framework, change of network capacity/congestion could deal with rate adaptation of a video encoder to ensure the best video quality and smooth playout. Rate adaptation in a video encoder would in turn trigger corresponding rendering strategies in the game engine and graphics renderer. On the other hand, rendering parameters could possibly help in a more efficient coding decision in fewer search steps. Metadata of video packets and game rendering allows the network to allocate resources more efficiently for smooth playout, graceful degradation, and low-latency delivery for action-based games.

DEVICE-AWARE GAME RENDERING In a cloud gaming service, clients’ devices are heterogeneous in display, processing capability, and power supply. They range from wide screen 3D-enabled TVs to power-constrained smartphones. As presented earlier, display resolution of a game video significantly determines the complexities of graphics rendering and coding, and bit rate for delivery. Adapting the video resolution according to the client devices provides

Rendering parameters Graphics renderer

Graphic images

Bitstream metadata Video encoder

Rate/quality control

Video bitstream

Network delivery

Supportable throughput/latency

Figure 4. Cross-layer design for joint rendering-coding-delivery optimization. large optimization space for energy efficiency of cloud servers and networks. Figure 5 illustrates device-aware rendering and coding for cloud gaming. For clients of HDTV with set-top boxes and wired broadband Internet, games can be rendered at the highest quality for maximum gaming experience without the concerns of limited power supply or a fluctuating wireless channel. For mobile clients with smartphones and wireless connections, video resolution can be reduced according to the subjective visual quality for mobile display and wireless channel quality. Frame rate and frame size can be adapted accordingly for different game types and device types. Such adaptation improves the energy efficiency of cloud gaming in massive multiplayer games.

BANDWIDTH-EFFICIENT SOCIAL GAMING The growing popularity of social networks demonstrates the desire of gamers to share their gaming achievement with their friends and family. Some game enthusiasts would be interested

SVC encoder 3D game scene Multiresolution video

Left view

MVC encoder

Multiview / 3D video Center view

Right view

Figure 5. Rendering and coding of game videos for various devices.


85


Green features of cloud gaming

Green designs of technical modules

The future

Software aspect: • Easier software maintenance • No compatibility issues at clients • Game binary is more secure from piracy Hardware aspect: • Higher utilization of hardware • More efficient central management of computing resources • Lower cost of computing resources • Less hardware upgrade/electronic waste Deployment aspect: • No upfront cost for clients •Lower sale and distribution cost • Copyrights of game developers are better protected

Data center: • Online monitoring and allocation of VMs •Scalability of computing resources according to demands Graphics rendering: • Optimization of rendering complexity • Power control of GPU via DVFS enumeration Multimedia compression: • Optimization of coding complexity • Power control of CPU via DVFS Network delivery: • Software defined networks • Massive MIMO • Cognitive radio networks • Heterogeneous networks

• Cross-platform, web-based HTML5 gaming • Cross-layer design: joint rendering-coding, coding-network, rendering-codingnetwork optimization • Device-aware game rendering: devicedependent adaptation of resolution • Social gaming: scalable bitstream for heterogenuous players/observers in social networks • 3D vision gaming: multiview/3D game video for high-end, action-intensive gaming

Table 1. Summary of cloud gaming. to watch the games play by other gamers, while some video gaming often involves multiple players with some observers. These players and observers are often heterogeneous in their device terminals. Scalable video coding (SVC) offers a competitive solution for this scenario, which would otherwise require transcoding or multiple encodings for various resolutions. SVC encodes a video into a layered bitstreamthat can be partially extracted and decoded for various resolutions and quality levels. SVC allows efficient utilization of cloud and network resources as duplication of multiple bitstreams of a game video is eliminated. In addition to bandwidth efficiency, SVC may also help in synchronizing game video display among massive numbers of gamers and viewers. Note that in massive multiplayer games, synchronizing among players is critical for an acceptable gaming experience.

EFFICIENT 3D VISION GAMING Video games with 3D vision are gaining popularity in high-end action-intensive games such as FPS and action-adventure. Depth perception in these games significantly enhances the gaming experience for clients with 3D-enabled displays like home TVs. Conventional stereo 3D vision requires two views. For multiview or free viewpoint 3D vision, additional depth maps are required for synthesis of intermediate views. Multiview 3D vision allows the “look around” effect, which is desirable for realistic gaming experience. However, it requires advanced multiview display with some computation capability at the client device for view synthesis. In natural videos, depth information may not be captured accurately, resulting in displacement of pixels in view reconstructions. In graphics rendering, however, accurate depth information is readily available from the rendering. Although renderings and coding of color and depth in multiview video coding (MVC) are linearly proportional to the number of views, the cloud gaming solution is more efficient in multiplayer games, where some rendering pipelines such as geometry processing are performed once, and then available for rendering and coding for multiple views of multiple players.

86

SUMMARY This article presents an overview of cloud gaming from the green media perspective. In addition to bringing high-quality games to thin devices, cloud gaming promises better efficiency in hardware utilization and software maintenance. Table 1 summarizes the green features and future challenges of cloud gaming discussed in this article. We discuss green designs of three major technical modules in cloud gaming: graphics rendering, video coding, and network delivery. In summary, energy efficiency of graphics rendering and video coding, which require heavy computations in GPUs and CPUs, can be improved via efficient algorithms with the DVFS framework. For network delivery, emerging techniques such as software-defined networks, massive MIMO, cognitive radio networks, and heterogeneous networks offer potential for green delivery of delay-sensitive massive data. We review some existing commercial cloud gaming services and introduce an open source testbed for the research community. Based on the existing platforms, we identify some promising directions in green design of cloud video gaming. Among them, HTML5, which is equipped with WebGL for graphic processing in browsers, is a promising web technology for a truly platform-independent solution for heterogeneous client terminals. Existing crosslayer optimization for multimedia communication can be extended to joint rendering-coding-network optimization for better efficiency of cloud video gaming. As heterogeneous client terminals have different display and processing capabilities, potential efficiency gain via device-aware game rendering should be investigated. Finally, green designs of emerging social gaming and 3D vision gaming are some other future research directions in this pursuit.

REFERENCES [1] C.-Y. Huang et al., “Gaminganywhere: An Open Cloud Gaming System,” Proc. 4th ACM Multimedia Systems Conf., 2013, pp. 36–47; http://doi.acm.org/10.1145/ 2483977.2483981.



[2] L. Liu et al., “Greencloud: A New Architecture for Green Data Center,” Proc. 6th Int’l. Conf. Industry Session Autonomic Computing Communications Industry Session, 2009, pp. 29–38; http://doi.acm.org/10.1145/ 1555312.1555319. [3] M. Armbrust et al., “A View of Cloud Computing,” Commun. ACM, vol. 53, no. 4, Apr. 2010, pp. 50–58; http://doi.acm.org/10.1145/1721654.1721672. [4] A. Beloglazov, J. Abawajy, and R. Buyya, “Energy-Aware Resource Allocation Heuristics for Efficient Management of Data Centers for Cloud Computing,” Future Generation Computer Systems, vol. 28, no. 5, 2012, pp. 755–68; http://www.sciencedirect.com/science/article/pii/S0167739X11000689. [5] T. Burd and R. W. Brodersen, “Processor Design for Portable Systems,” J. VLSI Signal Processing, vol. 13, 1996, pp. 203–22. [6] B. Mochocki, K. Lahiri, and S. Cadambi, “Power Analysis of Mobile 3D Graphics,” Proc. Conference Design, Automation Test Europe, Leuven, Belgium, 2006, pp. 502–07; http://dl.acm.org/citation.cfm?id=1131481.1131617 [7] J. Pool, A. Lastra, and M. Singh, “An Energy Model for Graphics Processing Units,” Proc. 2010 IEEE Int’l Conf. Computer Design, 2010, pp. 409–16. [8] L. Su et al., “Complexity-Constrained h.264 Video Encoding,” IEEE Trans. Circuits Sys. Video Tech., vol. 19, no. 4, Apr. 2009, pp. 477–90. [9] Z. He et al., “Power-Rate-Distortion Analysis for Wireless Video Communication Under Energy Constraints,” IEEE Trans. Circuits Sys. Video Tech., vol. 15, no. 5, May 2005, pp. 645–58. [10] W. Vereecken et al., “Power Consumption in Telecommunication Networks: Overview and Reduction Strategies,” IEEE Commun. Mag., vol. 49, no. 6, 2011, pp. 62–69. [11] J. Wu et al., “Distortion-Aware Concurrent Multipath Transfer for Mobile Video Streaming in Heterogeneous Wireless Networks,” IEEE Trans. Mobile Comp., to appear. [12] C. Hu et al., “On the Design of Green Reconfigurable Router Toward Energy Efficient Internet,” IEEE Commun. Mag., vol. 49, no. 6, 2011, pp. 83–87. [13] S.-P. Chuah and N.-M. Cheung, “Bandwidth Efficient Mobile Cloud Gaming with Layered Coding and Scalable Phong Lighting,” Proc. 21st IEEE Int’l. Conf. Image Processing, 2014, Oct. 2014. [14] M. van der Schaar and N. Sai Shankar, “Cross-Layer Wireless Multimedia Transmission: Challenges, Principles, and New Paradigms,” IEEE Wireless Commun., vol. 12, no. 4, 2005, pp. 50–58.


[15] M. Claypool and K. Claypool, “Latency and Player Actions in Online Games,” Commun. ACM, vol. 49, no. 11, Nov. 2006, pp. 40–45; http://doi.acm.org/10.1145/ 1167838.1167860

BIOGRAPHIES S EONG -P ING C HUAH received a B.Eng degree from Universiti Teknologi Malaysia, an M.S. degree from National Taiwan University, and a Ph.D from Nanyang Technological University (NTU), Singapore, all in electrical engineering. He was with Intel Penang Design Center, Malaysia, in 2004. In 2006, he spent a summer as a graduate intern at Taiwan Semiconductor Manufacturing Company (TSMC) Ltd in Hsinchu. He joined NTU Singapore as a research associate in 2008, working on a mobile media project. He is currently with Singapore University of Technology and Design (SUTD) as a postdoctoral research fellow. His research interests include cloud gaming, computer graphics rendering, image/video coding, wireless video streaming, energyefficient wireless networks, resource allocations, optimization and game theory. C HAU Y UEN received B.Eng. and Ph.D. degrees from NTU in 2000 and 2004, respectively. He was a postdoc fellow at Lucent Technologies Bell Labs, Murray Hill, New Jersey, during 2005. He was also a visiting assistant professor at Hong Kong Polytechnic University in 2008. During the period of 2006–2010, he was at the Institute for Infocomm Research, Singapore, as a senior research engineer. He joined SUDT as an assistant professor in June 2010. He also serves as an Associate Editor for IEEE Transactions on Vehicular Technology. In 2012, he received the IEEE Asia-Pacific Outstanding Young Researcher Award. N GAI -M AN C HEUNG is an assistant professor at SUTD. He received his Ph.D. degree in electrical engineering from University of Southern California (USC), Los Angeles, in 2008. From 2009 to 2011, he was a postdoctoral researcher with the Image, Video and Multimedia Systems group at Stanford University, California. He has also held research positions with Texas Instruments Research Center Japan, Nokia Research Center, IBM T. J. Watson Research Center, HP Labs Japan, Hong Kong University of Science and Technology, and Mitsubishi Electric Research Labs. His research interests are image and video processing, signal processing, and multimedia communication.

87