distributed video production over atm

0 downloads 0 Views 325KB Size Report
Central to DVP are distributed pilot applications for professional digital video ..... sent over ATM to a site where the BG is rendered, and compositing and ..... techniques for film and video archives for real-time restoration of impaired video (e.g..
DISTRIBUTED VIDEO PRODUCTION OVER ATM Manfred KAUL, Christian BREITENEDER, Simon GIBBS, Wolfgang HEIDEN, Dirk STEINBERG, and Markus WASSERSCHAFF GMD - German National Research Center for Information Technology IMK - Media Communication Institute Schloß Birlinghoven, D-53754 Sankt Augustin GERMANY http://viswiz.gmd.de/DVP, e-mail: [email protected]

ABSTRACT This paper presents an overview on and reports on early experiences of the European ACTS project AC089 called „Distributed Video Production (DVP)“ which started in late 1995. Central to DVP are distributed pilot applications for professional digital video production over ATM broadband networks (LAN and WAN). Distributed video production refers to situations where the cameras, recorders, switches, mixers and other equipment used in video production (or post-production) are located at several sites linked by high bandwidth network connections. The DVP project investigates requirements of broadcasters for several forms of distributed video production and runs a series of trials of distributed virtual studios, distributed rehearsals and remote video editing and retrieval. Together with North American partners a transatlantic broadband ATM link will be tested for distributed virtual reality simulations. This paper reports about two initial tests with a German public broadcaster and the German Telekom. DVP project partners are GMD and about 20 broadcasters, computer and video equipment manufacturers, and video production companies. More information can be obtained from http://viswiz.gmd.de/DVP.

1. INTRODUCTION The technological revolution of digital media is rapidly changing the world of broadcasters and media industry. In the emerging field of media processing we envisage computer science, communication and media technology coming together to explore new applications defining new products and new markets. These concerted technologies will enable media producers to remotely collaborate in digital video and multimedia production, post-production, archiving, indexing, and retrieval.

Distributed Video Production over ATM

-2-

ECMAST’96

Professional video production is inherently distributed: Video and computing equipment is physically distributed over several racks, rooms, studios and sites. Broadcasters increasingly outsource video production and post-production to specialized remote studios or upcoming virtual studios. Therefore there is an increasing demand for the enabling technology for distributed video production, which has been addressed in the European ACTS project AC089 called „Distributed Video Production (DVP)“ being started Sept. 1995 and planned for a 30 months duration. In the DVP consortium leading European broadcasters, computer, communication and media technology providers are joining their efforts in developing the technological basis for professional video production over ATM networks including remote contribution, archiving, editing and retrieval, post-production and dissemination of digital video. SGI and AVID providing leading computing platforms for broadcasters’ and in particular virtual studios’ needs are both sponsoring partners in the DVP consortium. Additionally, an American supercomputer center and a Canadian educational institute contribute distributed video applications and distributed virtual reality simulations. For these applications a broadband transatlantic ATM link will be used between the involved partners for real-time simulations.

TIME (yr)

2.5

PILOT APP

PILOT APP

Distributed Virtual Studio

Telepresence Applications

PILOT APP

Distributed VRSimulation

PILOT APP Distributed Video Archiving

1.5 T E S T O F B A S I C F U N C T I O N A L I T Y : Video Contribution, Retrieval

INTEGRATION

Digital Video Digital TransCoding and Standards: mission Standards Compression: CCIR 601... ATM / SDH MPEG II/Pro

IT-Components: Archives, Data Format, VR-Systems

Fig. 1 : DVP time scale and pilot applications The basic technology for DVP is transferring studio-quality digital video over broadband ATM networks. The distributed DVP architecture imposes stringent and complex requirements on the switching and transport architecture of a network. The different DVP applications face different problems of video coding and compression,

Distributed Video Production over ATM

-3-

ECMAST’96

transmission and processing delays, synchronization requirements, and quality of service (QoS) parameters. Each of these problems has to be solved for each application adaequately. E.g. delay is a critical issue in the distributed virtual studio application: An actor in a blue room will loose the physical coordinates if the video signal on the monitor is always a few seconds late. In order to solve this kind of problems under realistic conditions, pilot applications and trials of distributed virtual studios, distributed telepresence, distributed virtual reality, and distributed video archiving are substantial parts of the DVP project. The project identifies requirements and develops prototype applications for well defined, realistic scenarios in very narrow time frame (see fig. 1). The scenarios comprise a rich set of real life situations, a series of trials allows end users to judge the potentials of the services provided. Integrated systems and field trials are important elements of DVP. The prototypes developed in the project are not simply for lab use but will be used under actual working conditions. The DVP pilot applications are: • Distributed Video Post-Production: Editing and special-effects generation using both local equipment (editing console, DVEs) and remote equipment (video servers). • Distributed Video Production: Bringing together real actors and objects (props) from different - separated - studios in a common real or virtual studio. • Distributed Rehearsal: An immersive teleconferencing environment allowing small groups of actors and musicians at different studios to conduct rehearsals as if face-toface. • Distributed Video Archiving, Indexing, and Retrieval: Besides real-time applications like those mentioned above, some non-real-time applications also contribute to DVP. In Germany, additional motivation for the DVP trials stem from the introduction of ATM wide area networks (WAN) substituting the switched Video Broadband Networks (called VBN). German public broadcasters face a special situation these years: For almost ten years now, they have been using the 140 Mbps VBN with an additional 2 Mbps data link offered as a service of the German Telekom. The end-client interface only allowed for analogue video input and output with only one channel per VBN line, although the Telekom transmission has been digital. This service will be no longer available and has to be substituted by a new digital service, which will be ATM for sure. Therefore, German public broadcasters are very much interested in the DVP project for first results in using an ATM-WAN for distributed video production in their application field. This paper is organized as follows: Chapter 2 gives an overview on the architecture of DVP. The next chapter is more technical oriented towards MPEG for professional use and ATM. After these technical elaboration the following chapters describe in more detail two DVP pilot applications, the distributed virtual studio and the communication wall. They both have been developed and initially tested at GMD. The other DVP pilot applications will be described in a forthcoming paper. In the final chapter the role of DVP in the ACTS programme and its relationship to other projects is described.

Distributed Video Production over ATM

-4-

ECMAST’96

2. DVP ARCHITECTURE AND ORIENTATION The DVP project aims to take existing broadband technology, mainly ATM, as starting point and to develop services (applications) on top of ATM that directly address the needs of the television industry. Several broadcasters are included in the consortium and are providing user input and participating in field trials. Thus DVP is trial oriented with a strong element of user participation. Distributed video production refers to situations where the cameras, recorders, switches, mixers and other equipment used in video production (or post-production) are located at several sites linked by high bandwidth network connections (see figure 2). The DVP project investigates user requirements for several forms of distributed video production and is running a series of trials of distributed virtual studios, distributed rehearsal systems and distributed video editing and retrieval systems.

post production studio

TV studio 2

TV studio 1

DVP architecture on ATM

video server

TV studio & control room

Fig. 2: DVP ATM architecture for distributed professional video production The basic DVP architecture allows for integrating the different complex video production functionalities as „on demand“ services which can be ordered and used online over the network (see fig. 2). For professional video an acceptable video quality is prerequisite. Therefore the underlying network has to offer enough bandwidth for transmitting professional video signals, which might be up to 270 Mbps for D1. Shared

Distributed Video Production over ATM

-5-

ECMAST’96

video servers allow for random access of video shots and takes across the network. Post production studios with non-linear editing devices can access videos directly from the video server without file transfer and time consuming copying procedures. In order to exchange video, audio and data between the different clients and servers on the DVP ATM network, common interfaces, common video formats, audio formats, and data formats, have to be chosen. Uniform user interfaces would be of great advantage for operating the different applications. The distribution of functionality, processes, and computational power between server and clients have to be chosen carefully and depends on the application in a WAN or LAN networks. Basically the DVP project consists of two layered development and field test scenarios: • a wide area ATM testbed (including trans-Atlantic links) connected to digital video production test sites with basic user functions as prerequisite for broadband distribution of digital audio/video applications. • distributed video production pilot applications (e.g. a distributed virtual studio or a distributed video editing or simulation system). 2.1 Key Issues Professional distributed video production impose very ambitious and complex requirements on the switching and transport architectures of the underlying network. New transport layers like ATM imply that the video signal will no more be carried in continuous streams, but will be broken into pieces and packetized into cells. The characteristics of the underlying transfer system are quite different than with isochronous D1 streams, quality of service aspects like cell loss and jitter will be important. But packetization offers the chance that video distribution is no longer limited to local distribution. In detail, the following questions will be answered during the DVP trials: • How can ATM-connectivity based on SDH (STM1 to STM4) be used for connecting CCIR 601 studio systems? • Transmission and processing delays: what is the impact of such delays during video production, what are acceptable delays, where is isochronous transmission really required? • Synchronization: what are the synchronization requirements for tightly-coupled distributed video processing? • What quality of service of the transmission system is required? MPEG-2 compression has been designed so far mainly for program distribution at relatively low data rates (8 MBit/sec) per stream. There is an upcoming new MPEG-2 variant for professional use called MPEG 4:2:2 profile which is better suited for the video production chain (contribution, production, post-production and distribution), because

Distributed Video Production over ATM

-6-

ECMAST’96

these streams consist of I-frames only, allowing for non-linear access to every frame (see chapter 3). But there are still open questions regarding the influence of compression and concatenation on the video quality in the different production stages. On the other hand, the usage of compression offers the choice to get video transferred online to places which could not be interconnected before. The DVP project makes use of existing technology in the areas of video encoding, storage and transport systems as far as possible. It also builds upon existing work in areas such as non-linear editing, video servers, and virtual studios. Only not yet existing components needed in the DVP trials are developed. Partners from broadcasting industry participating in a European network profit from access to DVP facilities for a variety of reasons: • reduced production costs and shorter production times • integrated training and documentation facilities • better (tested/simulated) product quality • integrated production and distribution processes In summary, DVP will build advanced communications services that fit the needs of a particular user group. These services are used in field trials and contribute to European economic development and social cohesion.

3. DIGITAL VIDEO OVER ATM Professional digital video equipment (according to CCIR 601) operates at bitrates up to 270 Mbit/s without any compression. For economic reasons a reduction of the required bandwidth for transmission to remote sites is imperative, even with high speed transmissions technologies. This can be achieved by a number of compression techniques which are available today or being investigated. 3.1 Requirements for video compression schemes One feature of a good compression technique is a high compression ratio while maintaining good picture quality. Usually there is a clear trade-off between these two factors. The various application scenarios of the DVP project make more demands for video compression: 1. Today’s broadcasters and video producers will accept only very high picture quality standards. 2. Every production chain consists of multiple steps, where video material is copied, cut or manipulated with digital video effects. In digital environments this leads to

Distributed Video Production over ATM

-7-

ECMAST’96

repetitive code/decode-cycles (generations). Since most video codecs are not lossless, picture quality may suffer from the introduction of multiple generations. A „good“ compression method therefore should be robust against these effects. 3. Our virtual studio application uses chroma keying for overlaying foreground and computer generated background. Acceptable results can only be achieved if the blue room signals have stable color characteristics, even after multiple generations. 4. It turns out that due to the distributed nature of our applications, delay is very critical and has to be minimized. Therefore it is unacceptable that compression codecs introduce more delay than a few milliseconds, or synchronisation may become hard or even impossible. 5. Coding of both video and audio simplifies synchronisation as opposed to the use of separate channels. 6. Finally, such a method should be adopted by an international standardization body in order to ensure interoperability (as opposed to proprietary standards, i.e. M-JPEG). In the past, ETSI video codecs (ETS 300 174/ITU-R 723) have been used successfully in our tests for transmission of digital video over ATM networks. This solution offers some advantages: First of all, there are commercial and reliable products available. In initial tests (see chapter 6) it turned out that these ETSI codecs kept the induced delay down to less than 20 milliseconds. On the other hand, the ETSI coding scheme is quite inflexible with respect to different bitrates and incorporates no standard mechanism for audio coding. 3.2 MPEG-2 and the 4:2:2 Profile The MPEG-1 [1] video compression was designed with the immediate goal of standardizing video and audio for compact discs. Its successor, MPEG-2 [2], defines a syntax for the efficient representation of broadcast video. These broadcast and distribution applications require support for interlaced video and more audio channels like surround audio. This generic standard has been designed to support a wide variety of applications, ranging from home video applications, professional production and distribution up to HDTV applications. Therefore, a number of levels and profiles have been defined, which constrain the allowed values for parameters like picture resolution, maximum bitrate, chrominance subsampling etc. This two-dimensional division into levels and profiles facilitates implementation of codec soft- and hardware, because only those features have to realized that are allowed at an specific level/profile. So far defined profiles have been oriented towards storage and distribution of compressed video. 3.2.1 Fundamentals of MPEG-2 Video Compression MPEG-2 is a block-based coding scheme and achieves compression by exploiting spatial redundancy in a single frame (intra frame coding) as well as temporal redundancy

Distributed Video Production over ATM

-8-

ECMAST’96

between frames (inter frame coding). Intra frame coding uses DCT and variable length coding in a fashion quite similar to the JPEG [3] still picture compression standard. Temporal redundancy reduction is based on the assumption that parts (square blocks) of the picture can be represented as a translation of corresponding parts from a previous picture. This motion-compensated prediction is computationally complex since a block’s neighborhood (in the successor) has to be searched for a matching block. Thus MPEG is an asymmetric algorithm. MPEG introduces three types of frames: Intrapictures (I), predicted pictures (P) and bidirectionally interpolated pictures (B). Intrapictures can be decoded without reference to other pictures and provide entry points for random access, while predicted pictures are coded with reference to previous I- and P-frames. B-frames have the highest amount of compression potential but require both a previous and future frame for prediction. Figure 3 shows a sample sequence of I-, B- and P-frames, and illustrates references between predicted and bidirectional frames. From this figure it becomes obvious, that B-frames cause delay because of their forward references and increase memory requirements in the decoder. B id ire c tio n a l P re d ic tio n

I

B

B B

P

B

B

P

B B

B

P

B

B

F o rw a rd P r e d i c t i o n

Fig. 3: A typical sequence of frames in MPEG-2 I-frames in a bitstream can be decoded independently from other frames and thus constitute recovery points in case of transmission errors as well as random access points. Since different frame types can be grouped quite flexible, it is possible to trade off application requirements against bandwidth demands. When MPEG-2 is used for distribution of digital video, less I-frames would be inserted because B- and P-frames achieve higher compression. On the other side if random access is essential, for example in non-linear editing applications, one would use I-frames, only. For distribution purposes the so-called main profile at main level (MP@ML) is the most important, because it provides all the syntax and quality requirements for current TV broadcasting. But it is not well suited for studio applications, since (a) it is not possible to achieve editable high quality video with I-Frame only coding at 15 Mbit/s; (b) 4:2:0 chroma subsampling is not sufficient for professional applications (i.e. digital video effects) and (c) multiple generation behavior of MP@ML is not robust enough for professional productions. To remedy these limitations an amendment to part 2 of MPEG-2 was created, that describes a new profile, the so-called professional profile or 4:2:2 profile [4]. Key requirements were: Provision of excellent video quality, even after multiple coding

Distributed Video Production over ATM

-9-

ECMAST’96

generations, support for 4:2:2 chrominance subsampling, backward compatibility with main profile, editibility of the compressed bit stream and flexibility. Compared to main profile, the 4:2:2 profile allows bit rates of up to 50 Mbit/s and 4:2:2 chrominance subsampling. 3.2.2 Studies of 4:2:2 Profile Performance Parallel to the standardization efforts a number of tests have been made to verify its suitability for professional applications. Further results on the performance of the 4:2:2 profile are reported in [5]. There, test sequences (i.e. mobile & calendar) have been encoded and decoded eight times to investigate multi-generation behavior. Digital video effects were simulated by shifting frames between two encoding generations. Additional test were carried out with chroma-keying applications. The results from these experiments can be summarized as follows: • 4:2:2 profile provides excellent picture quality, even after eight generations of encoding/decoding. • Performance in chroma-keying experiments was excellent at 50 Mbit/s and very good at 30 Mbit/s. These results correlate with preliminary tests carried out at GMD in the framework of the DVP project. To sum up, MPEG-2 4:2:2 profile satisfies all the requirements mentioned at the beginning of the section. Since there is no suitable hardware available so far, it was not possible to measure the delay introduced by ML@422 profile codecs. But, in the case of I-frame only coding, we expect a quite acceptable delay. 3.3 Advantages of ATM ATM is clearly the emerging standard at least for broadband networks. Features of ATM technology which are most beneficial for distributed video applications are: • Deterministic Bandwidth: Ability to allocate bandwidth for distributed video applications and guarantee quality of service regardless of what other applications are running on the network. • Scalability: ATM bandwidth is scalable to meet the requirements of expanding application complexity and faster host processors. The usage of video compression offers the choice to get video transferred online to places which could not be interconnected before. New transport layers like ATM imply that the video signal will no more be carried in continuous streams, but will be broken into pieces and packetized into cells. The characteristics of the underlying transfer system are quite different than with isochronous D1 streams, quality of service aspects like cell loss and jitter will be important. But packetization offers the chance that video distribution is no longer limited to local distribution.

Distributed Video Production over ATM

- 10 -

ECMAST’96

The key requirements for distributed video applications that should be supported by the underlying ATM systems are (a) bandwidth allocation (from 3 up to 270 Mbps for uncompressed D1 video signal, burstiness 1-10) (b) appropriate performance parameters, such as cell delay, mean cell transfer delay, cell delay variation, cell error ratio, BER, cell loss rate, cell misinsertion ratio, and synchronization. Constant end-to-end delay has to be guaranteed in spite of jitter and cell delay variations. The demand for continual real-time delivery of the bit stream prevents the possibility of re-transmission of error packets.

4. PILOT APPLICATIONS The four major pilot applications in DVP are: Distributed Virtual Studio: One of the key applications of the DVP project is the distributed virtual studio (DVS). The term "virtual studio" refers to real-time 3D graphics systems used to render a virtual set in sync with live camera motion. For more details see chapter 4.1. Distributed Video Editing and Retrieval: The main goal of this application is to provide an integrated environment in which new technologies for video and audio post-production (digital video archive, and non-linear editing) will be accessible via a uniform user interface, disregarding their physical location. The video editing functionalities will be coupled with a powerful Information Retrieval system in order to access the distributed video archives in a transparent and effective way. The three integration levels (data, control, and presentation) will be achieved exploiting the telematic services provided by basic functionalities on top of the ATM network technology. Distributed Rehearsal: This task of the DVP project targets in developing a telepresence environment for the support of Distributed Rehearsals. These rehearsals can be theater rehearsals, musical rehearsals, or even television soap opera rehearsals (assuming of course these last ones exist!). The aim of the distributed rehearsal application is to allow the organization of rehearsals without requiring the participants (actors, musicians, conductors etc.) to be physically in the same room, eliminating thus the need to travel from one city to another for participating in the rehearsal. Distributed VR Environments: The goal of this workpackage is to establish an interactive, real-time ATM network connection between two Virtual Reality visualization systems across the North Atlantic. In the following two chapters we describe two pilot applications in more detail.

4.1 Distributed Virtual Studio The idea of a virtual studio originates in attempts to overcome the constraints of traditional chroma-keying. A major problem with chroma-keying and other layering

Distributed Video Production over ATM

- 11 -

ECMAST’96

techniques used in video production is that camera motion in the foreground (FG) layer (for example, the weather announcer) is uncorrelated with the background (BG) layer (for example, the weather map). In other words, if a pan or zoom takes place it may not be matched with changes in the background. The result is that spatial relationships existing between the two layers, and the viewer’s perception of the layers forming an integrated whole, are altered or lost. In traditional chroma-keying situations, such as a weather announcer and map, the above problem is overcome by "locking off" the camera and using a static background. This of course is very limiting and a number of techniques have been developed which extend the usefulness of chroma-keying. With a virtual studio, the static background is replaced by a dynamic, computergenerated, three-dimensional background. The foreground camera is then free to move but must be "tracked" so that the background can be generated with the proper perspective. With the background no longer a static image, the real advantages of virtual studios become apparent. As in real studios, people before the camera are free to move. Typically production takes place in a blue room - a uniformly lit and uniformly colored room allowing easy replacement of the blue background with a synthetic background. Since the new background is computer generated it is very easy to modify - walls can be moved, their color changed, new textures applied etc. It is possible to make these changes both during a production or between productions. Furthermore, virtual studios require no storage space (other than as bits in files) and can be easily transported (over communication networks rather than in trucks). The basic components of a virtual studio are a FG generation system (camera, tracking system, blue room), BG generation system (renderer), FG+BG compositing system, and interfaces to other studio equipment such as recorders and mixers. A distributed virtual studio results from placing these components over two or more sites. Many different scenarios are possible – differing in terms of complexity, synchronization requirements and communication requirements. Scenarios that will be tested in DVP trials include: remote rendering – a remote site receives studio camera tracking data over ATM, and returns the BG signal also over ATM; remote blue room – FG signal and camera tracking data are sent over ATM to a site where the BG is rendered, and compositing and recording are performed. Distributed virtual studio production offers several advantages as compared to a "local virtual studio". First, expensive equipment (e.g., the graphics supercomputers) can be shared between many studio sites. Second, productions of different sizes (e.g., two camera, six camera) can be supported by appropriate network configuration, And third it opens the way for new digital services such as virtual set providers. However a distributed virtual studio production makes heavy demands of the underlying network – very high reliability is needed, particularly for live productions; “D1” quality video; sustained high data rate transfers (e.g., multiple digital video streams for several hours); very low jitter; and very low latency. The last requirement in particular should be stressed: if the network/codecs introduce significant delays, camera operation becomes increasingly

Distributed Video Production over ATM

- 12 -

ECMAST’96

difficult (e.g., the FG and BG layers become “decoupled” during fast camera moves). Because of these stringent demands, the distributed virtual studio is considered a key application for testing network services within the DVP context.

4.2 Distributed Rehearsal: The Communication Wall The DVP project is concerned with all user requirements of the TV buisiness for distributed high-quality digital video production over broadband networks. Therefore DVP includes TV productions in separate dislocated studios with the goal of generating the impression that all participants meet at the same place. Clearly, for broadcasters’ needs, video conferencing and desktop video conferencing is not enough for this purpose. The principal idea of the Communication Wall installed at GMD is that a real room is equipped with a wall-sized display (the Communication Wall) onto which a "virtual" extension of the room can be projected. Into this virtual extension video images of remote participants are blended, such allowing perceiving participants in life-size, almost as if face-to-face. As the local participant moves their location is tracked, thus allowing the synthetic scene to be rendered with the correct perspective. Ultrasonic and electromagnetic sensors were tested to find the best fit to the application requirements, but also to the specific features of the display area.

A’s real office

participant A

A’s virtual extension

participant B

wall-sized display

Fig. 4: The Communication Wall The display room in operation covers an area of 3m times 3m and a height of 2.25m. It resembles a box of which one surface has been removed to allow people to enter or view the projection from outside (see figure 4). The wall opposite of the missing wall is entirely covered by a display surface. The current setup is an office with a desk, chairs, light sources, speakers and a desktop camera. Behind the display surface a projection area is

Distributed Video Production over ATM

- 13 -

ECMAST’96

required to create the necessary distance between projectors and display surface (approx. 5m). The environment also allows for stereoscopic rendering and viewing with passive glasses. However, a primary concern is that no special sensors need be attached or worn by participants for videoconferencing. Stereoscopic rendering is employed when the communication aspect of a session is dominated by the requirement to cooperate in a 3D space. Conference participants have to be extracted from the background before they are mapped as video textures to a transparent grid and positioned in the rendered scene. This separation is currently performed by traditional chroma-keying of video, shot in a blueroom. However, since it cannot be expected that conference participants have a blue-room at their disposal, we are working on real-time extraction out of a static background. Under the assumption that the camera position is fixed we shoot a frame of the room without any persons present. This image (or the average of a sequence of images) is used as a reference frame to compute differences to frames when persons are present. The Communication Wall is certainly not intended to replace face-to-face meetings, but to offer an alternative when such meetings are difficult or impractical. Scenarios where a Communication Wall could be used include: • distributed negotiation - sensitive negotiations and conflict resolution meetings are best held face-to-face; the telephone or traditional forms of video conferencing fail to convey body language and other subtle channels. When it is simply not possible to gather participants in a single place, a Communication Wall may be an appealing alternative. • the physically disabled - for such people the Communication Wall offers a means of surrogate travel, a natural way of communicating with other people despite being in different places. • education - teachers or professors could hold office hours with students located in other cities or countries. Trials within the DVP project using the Communication Wall focus on distributed rehearsing and production planning in the TV domain. Actors and people from the TV production stuff on remote locations can watch and discuss scenes used in a future production. Actors can rehears by not only seeing, talking to and watching each other but also get the right perception of a common space. A more detailed discussion of the Communication Wall can be found in [6].

Distributed Video Production over ATM

- 14 -

ECMAST’96

5. TRIALS In this chapter we give a brief description of trials planned an DVP and go into more details describing the early tests with a German public broadcaster and the German Telekom. 5.1 Planned Trials The DVP project aims to take existing broadband technology and develop services (applications) that directly address the needs of the television industry. Several broadcasters are included in the consortium and have committed to providing user input and participating in field trials. Thus DVP has a strong element of user participation. Network techniques and topologies for integrated services networks enabling distributed programme production for multimedia services and interactive TV are evaluated. Quality of Service (QoS) aspects and adaptation of network techniques (ATM) to the high quality studio requirements for studio production are investigated. DVP uses WWW to disseminate public results of trials. The URL is: http://viswiz.gmd.de/DVP/Public.

5.2 Two Early Tests In the early stage of the DVP project two initial tests have been carried out in order to demonstrate the feasibility of the DVP pilot applications and to gather some early experiences with distributed setups in the broadcasting domain. 5.2.1 First Test: Distributed Virtual Studio with a German public broadcaster On November 26, 1995, SWF (a German broadcaster), in cooperation with GMD, Deutsche Telekom and SGI, recorded the award ceremonies for Video Only (an international video art competition) in a virtual studio that was delivered in real time over ATM. Live camera tracking data from the recording site – a blue room studio in BadenBaden – was sent to GMD in Bonn over 300 km away. Using an SGI Onyx/RealityEngine2, GMD rendered the virtual studio in sync with the remote camera motion and sent back a broadcast quality signal for both the background and the accompanying mask to SWF over a SDH/STM1 ATM connection. The virtual studio was then chroma-keyed with the live camera signal and recorded. In addition, to compare performance and quality, a second camera was connected to a local virtual studio system. This was, to the authors’ knowledge, the first use of a distributed virtual studio for a broadcast video production. The configuration of the involved video components used for the Baden-Baden test is shown in figure 5 below.

Distributed Video Production over ATM

SGI Onyx RealityEngine2

- 15 -

ECMAST’96

ATM switch (SWF/Baden-Baden)

Sirius video Sirius video BG

mask

ETSI encoder

ETSI encoder

mul tiplexor/ line adaptor

ATM switch (GMD/Bonn)

line adaptor demul tiplexor/

ETSI decoder

ETSI decoder

frame synchronizor

frame synchronizor

BG

mask

camera

delay FG

video sync generat or

video sync generator

155 Mbps ATM (Deutsch Telekom)

compositor

FG+BG recording, mixing, monitoring

Fig. 5: Video connections for Baden-Baden test The video equipment in the Baden-Baden test generally accepted serial digital video (CCIR 601 4:2:2) connections with a data rate of 270 Mbps. Each ETSI encoder compressed the 270 Mbps 4:2:2 data stream to a 34 Mbps E3 signal. A E3/ATM terminal adapter was used to transmit these 34 Mbps streams over 2 ATM virtual channel connections (VCCs) of 40 Mbps each, using AAL-1 constant bitrate circuit emulation yielding a total load of 80 Mbps on the ATM link. Tracking data was sent via a separate 32 kbps VCC within the same 155 Mbps ATM connection.

Fig. 6: From the SWF production: dancers in the blue room The Baden-Baden test successfully demonstrated the viability of remote rendering of a virtual studio via ATM. During the nearly two hours used for the production (which was

Distributed Video Production over ATM

- 16 -

ECMAST’96

broadcast after editing) there were no network problems, the remote signal was stable and of the same apparent quality as the local signal, and the additional delay introduced by the network was negligible (ATM delay from Bonn to Baden-Baden was 4 ms). 5.2.2 Second Test: International Press Conference of German Telekom Bonn

Fig. 7: Telekom test scenario Between January 31 and February 2, 1996, the German Telekom held its annual International Press Conference (IPK) at Bonn. Over 250 journalists from over 17 countries attended. Several demonstrations showed new and innovative applications in the field of telecommunications. GMD and its partners demonstrated the distributed virtual studio with a blue room at the Telekom building and using the Telekom ATM pilot network to access the remotely rendering supercomputers at GMD. The setup was a little bit different to the one used in Baden-Baden. This time, the camera video signal was transmitted along with the tracking data to GMD, where both the rendering and the chroma-keying took place. The final video signal was transmitted back to the blue room. Therefore a bandwidth of 40 Mbps in each direction was needed rather than 80 Mbps in one direction. This had the advantage that the bandwidth requirement was symmetric with the maximum required bandwidth being half, but the critical thing about this setup was that the ETSI-compressed signal was used for chroma keying, which is very sensitive to color distortions. In addition, the camera signal was compressed twice with the final recording being performed at the blue room site.

Distributed Video Production over ATM

- 17 -

ECMAST’96

In these first DVP tests we used constant bit rate channels because of the ETSI codecs. In future field tests and trials to be done within DVP, experiments with variable bit rate encodings and transmissions (e.g. M-JPEG, MPEG) are planned.

6. CONCERTATION WITH OTHER ACTS PROJECTS Bridging the gap between video and ATM technology, the DVP architecture is the basic platform for distributed video applications based on ATM broadband networks for professional production use. The objective for the distributed DVP architecture is to enable TV producers to improve the economic efficiency and the time to market aspect of their broadcast program productions. For production environments a way of taking advantage of the basic ATM service layer available through ATM connectivity will be that advanced video production techniques will be accessible as an external service. By this means, ATM connectivity allows a "studio on demand" scenario, which enables TV producers to save equipment, and even more important, training and manpower costs. On the other hand, a new service industry is expected to develop, when new information products like virtual sets are being contributed in real time from an outside service provider to the program producer. In DVP trials distributed virtual studios, virtual reality and distributed editing and retrieval facilities are interconnected via ATM, as described above. But DVP is not a closed architecture. In the framework of the ACTS programme several other projects are interrelated to DVP developing new video creation and manipulation techniques, which hopefully will enter a commercial phase turning their scientific and technical achievements into professional services. These services will be offered locally offline, but within the DVP architecture could also be offered online via the DVP ATM-LAN or -WAN network. The ACTS projects which are most closely related to this approach are the following: PANORAMA, SICMA, MIRAGE, AURORA and RESOLV. Areas of mutual interest occur with other European projects, such as the ESPRIT Euromedia project, and with American activities such as the SGI/Sprint Drums service. • PANORAMA (ACTS 0078) enhancing 3D telepresence using offline image analysis using 3D Model objects, vector coding and interpolation of intermediate views, multiview cameras autostereoscopic displays. • SICMA (ACTS 0119) designing a cost efficient scalable DAVIC compliant server for the delivery of images, data and continuous multimedia information and to demonstrate its efficiency by applying it to a relevant application, namely the so called Virtual Museum. • MIRAGE (ACTS 0165) exploring and developing a low-cost platform (Virtual Edit Suite) for the creation and manipulation of virtual, both real and computer generated, environments in the studio using motorized camera memory heads and interfacing random access image storage/replay devices.

Distributed Video Production over ATM

- 18 -

ECMAST’96

• AURORA (ACTS 0202) developing highly sophisticated automated restoration techniques for film and video archives for real-time restoration of impaired video (e.g. video scratches, impulsive and white noise, film grain, drop-outs, moiré patterns, crosscolor, blur, missing images, geometric distortions, sound impairments). • RESOLV(ACTS 0286) producing a cost-effective system for the construction of 3D models of the environment using a mobile unit that includes a laser range scanner and video camera that pan and tilt to capture panoramic range and intensity data on the surroundings. The surface and texture information will be used to generate virtual views of real locations in order to enhance the sense of 'being there' in telepresence and virtual reality applications. The same system can be used to speed up the capture of dimension and surface information on buildings for construction applications.

AURORA video restoration

DVP Distributed Virtual Reality

DVP Distributed Virtual Studio

PANORAMA 3D telepresence

DVP Distributed Editing/Retrieval

DVP architecture on ATM

SICMA DAVIC compliant video server

DVP Distributed Rehearsal

MIRAGE Virtual Edit Suite

Fig. 8: Integrating remote services developed by ACTS projects into the DVP architecture Most mentioned ACTS projects are devoted for local use at the sites where they are needed most urgently, e.g. at INA near Paris for the AURORA service, because it will improve the quality of the local archive at INA. Of course, these services are also of interest for other video and multimedia production scenarios. Ideally, they should be available online „on demand“ for the whole production industry. The appropriate network architecture would be the DVP ATM network architecture, which has been tested inside the DVP trials for the described video production services and are planned to be extended

Distributed Video Production over ATM

- 19 -

ECMAST’96

to the other pre- and post-production services developed by the other current projects within the ACTS programme.

7. CONCLUSION In the DVP project, the enabling technology for tomorrow’s distributed video production settings are investigated and brought to the user. The broadcasting industry will benefit in a number of ways. Production costs can be reduced and productions times shortened. The real strength of the project is its strong user participation, that results in a better understanding of their real requirements and helps to solve real-world problems. The experience from initial tests encourage us that the trial-driven approach leads to competitive distributed application for video production. Professional quality video production impose especially ambitious and complex requirements on the ATM switching and transport facilities, that are explored and tested in the different DVP pilot application.

REFERENCES [1]

ISO/IEC 11172-2 Information Technology - Coding of moving pictures and associated audio for digital storage media at up to about 1.5 Mbit/s - Part 2: Video, 1993.

[2]

ISO/IEC 13818-2: Information Technology - Generic coding of moving pictures and associated audio information - Part 2: Video, 1994.

[3]

ISO/IEC JTC1 Committee Draft 10918-1. Digital Compression and Coding of Continuous-Tone Still Images. Part 1. Requirements and Guidelines. Feb. 1991.

[4]

ISO/IEC 13818-2:1995/DAM 2: Information Technology - Generic coding of moving pictures and associated audio information - Part 2: Video. Amendment 2: 4:2:2 profile, 1995.

[5]

Ricken, C. The 4:2:2 Profile of MPEG-2 For Use In A Studio Environment. Proc. IBC’95, S. 542546, Amsterdam, 1995.

[6]

C.J.Breiteneder, S.J. Gibbs, and C. Arapis: TELEPORT - An Augmented Reality Teleconferencing Environment; Proc. 3rd Eurographics Workshop on Virtual Environments Coexistence & Collaboration, Monte Carlo, Monaco, Feb. 1996.