videowhiteboard: video shadows to support remote collaboration

1 downloads 0 Views 1MB Size Report
[email protected].tom,. (415)494-4353. ABSTRACT .... [email protected].tom,. (415)336-1636. 315 ...... MA: Addison-Wesley,. 1982. Lauwers,. J. Chris and.
VIDEOWHITEBOARD: VIDEO SHADOWS TO SUPPORT REMOTE COLLABORATION John Scott System Xerox

C. Tang* L. Minneman

Sciences

Palo 3333 Palo

Alto

Laboratory Research

Center

Coyote Hill Road Alto, CA 94304

[email protected],

(415)494-4359

[email protected],

(415)494-4353

ABSTRACT

VideoWhiteboard remote shared

is a prototype drawing activity.

whiteboard-sized orators

who

each user

shared

in remote

of collaborators

development empirical

of studies

including shared

sites.

at the

remote

VideoWhiteboard

is

experiences

in

support

It allows

and a shadow

of collaborative

drawing

to

It provides a space for collab-

drawing

are located

to see the drawings

gestures

tool

The

based

drawing

using

prototype.

of the

site.

the

on

activity, VideoDraw

VideoWhiteboard

ables remote collaborators to work if they were sharing a whiteboard,

en-

together much as and in some ways

allows them to work together even more closely if they were in the same room.

than

Figure

1: Scene from

Sharing KEYWORDS:

gesture,

systems,

collaborative

video,

user interface,

shared

design

drawing,

process.

a common

resource

needed

nication

between

remote INTRODUCTION

Over

two

thousand

began

entertaining

plays

[March,

brightly

that

ago,

the imperial

the rear

shadows

years

1938].

colored

against

flat

This

court form

puppets

surface

Chinese with

of

that

of a backlit

in

artisans

are screen,

can be seen by the audience

Over

casting

in front

puppets

are

An Indonesian

granted

to

provided

copy that

without

fee

the copies

all or part are not made

of this

et

of

commercial

advantage,

other.

design

who

This

project

commu-

are

need

in a study

physically

for

a shared

of collaborators

that

sites [Olson

or distributed

last

decade,

that

partially

[1979]

collaborators.

1. to

was

& Bly,

distributed in press].

reported

blackboard

several

systems

address on for

the

this

have

need.

development

teleconferencing.

been

O’Boyle of an Many

Collaborators

who

are separated

by

geographical distance tell tales of sending faxes back and forth while talking on the phone in order to have timely interaction over graphical Shared window systems that enable information. people to interact over a common view of text and

is for

notice and the title of the publication and its data appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, raquiras a fse and/or specific permission. e 1991 ACM 0-89791 -383 -3/91 /0004 /0315 . ..$1.50 direct

each

is an important

graphical

current video teleconferencing facilities include overhead cameras or video copy stands for presenting images of drawings to remote

barely

material

al.

the

electronic

variety

of this art form, wayang kdit, is shown in Figure VideoWhiteboard uses a similar shadowy effect help remote collaborators work together. Permission

space

interactive

collaborators

two remote

developed

the

manipulate

to the audience.

play performance

uses

are

to

a software

pressed

project distinct shadows shadows of the rods that

used

drawing

for

space was noted

between

shadow

drama

the screen. The puppets onto the screen while the perceptible

from

drawing

a shadow

the ACM copyright

“Current address of first author is: John Tang, Sun 2550 Garcia Avenue, Mountain Microsystems, 94043, [email protected], View, California (415)336-1636. 315

graphics

through

a computer

network

become review).

available These

(see [Lauwers approaches,

& Lantz, however,

clumsy or limited in their interaction over drawings familiar

with

have

ability to support fluent in a way that people are

from face-to-face

collaboration.

VideoWhiteboard is one of a series of prototype that support collaborative drawing activity natural

and familiar

also

19901 for a are often

way.

It uses video

tools in a

technology

to connect whiteboard-sized drawing surfaces It provides a shared between remote locations. “virtual whiteboard” that allows collaborators to interactively others’

create

gestures

marks

in relation

and see shadows

of each

to those

Remote

marks.

Figure

2: Schematic

Figure

3: User’s

of 2-person

VideoDraw

collaborators can draw on, erase, and gesture at the VideoWhiteboard screens much as if they were The interacting around a shared whiteboard. development empirical [Tatar,

of VideoWhiteboard

is based

studies of collaborative 1989] and is closely

VideoDraw

prototype

[Tang&

drawing related

Minneman,

This paper describes VideoWhiteboard process by which it is being developed. with

a review

of VideoDraw

and

describe

on our activity to the

1990]. and the We begin how

our

experiences with VideoDraw led to the development of VideoWhiteboard. Then we describe what VideoWhiteboard is and report on early observations of Finally, we discuss some issues people using it. raised

in these preliminary

of VideoWhiteboard, prototype and about in general.

observations

both about collaborative

the design of the drawing activity

in Figure

2.

It consists

each other’s marks and the hand made in relation to those marks. view of a VideoDraw

collaborative representation

is shown

in Figure

screen

drawing prototypes. An of this development process

4. An interdisciplinary

working

videotape records of face-to-face collaborative [Tang et al., 1990]. The analysis focused on

drawing activity, from this analysis

of an

Observing

interconnection of cameras aimed at video display screens. Users draw on the video display screen (using dry erase whiteboard markers) and those marks and accompanying hand gestures are imaged by the camera and displayed on the other screen. This arrangement creates a composite shared drawing surface where the collaborators can see

typical

of other idealized

study work

[Tang& Minneman, 19901 is a prototype enables collaborators to share a video A schematic diagram of a 2-person is shown

of a VideoDraw

group of anthropologists, computer scientists, and designers applied interaction analysis methods to

EXPERIENCES WITH VIDEODRAW VideoDraw tool that sketchpad. VideoDraw

view

of the use

how

and a subset of the observations led to the design of VideoDraw. people

used

VideoDraw

and

com-

paring it to how people work face-to-face led to the development of VideoWhiteboard. Although this idealization implies a linear sequence of steps, the actual process involved alternating between looking at various kinds of collaborative work (face-to-face, using VideoDraw, using other shared drawing tools) and modifying the designs of VideoDraw and VideoWhiteboard.

gestures that are Figure 3 shows a

screen. Our observations of people using confirmed that they often used hand

While the design of VideoDraw was informed by studies of collaborative drawing activity, studying VideoDraw in use also contributed to a better understanding of that activity and the development

VideoDraw gestures to

enact simulations or mediate their interaction, and that these gestures were often made with respect to a referent sketch in the drawing space. Using

316

Figure

4: Iterative

development

of tools to support analyzing

VideoDraw,

remote

collaborators

were

the process of creating and referring (rather than just seeing the resulting which

is an important

activity.

resource

Collaborators to be drawing

VideoDraw

drawing

surface.

and building

able

to see

difficult

to drawings drawings),

in shared

which cannot together over

Furthermore,

drawing used at the

surface provides than, for example,

computational

to align

were drawn). made on their

be accoma single

VideoDraw

through

marks

helped

several

The

limitations

relatively

in the

small

video

request

others

to erase their

upward

facing

CRT

viewing ifications

problem. to the

observations

informed

\ .. \

VideoWhiteboard

use of

Straddling at the same

to suggesting VideoDraw, of design

the

an time

modthese Video-

is

,’,

/“

,. :,:. .n .. .: .:,: :.:.:. .,.,.:

~

J

t

video projector

Figure

\,’.

:.:,:.

II ,.”

..,,, .....

.

\,’. 1. .

,: .:.:

I

5: Schematic

Ii,: .:..

tool

k ....: .::.. ,,

l-l ....:. ...... ,.,.,. .:.:.:

317

.,

x,

::::.: .:,.,. .:. :.:,,,

system

. =

;:.:.:.j

of VideoWhiteboard

o

.

/“

.

...... .:.:.,

/’

1

. .

..:. :.:. ., ....

/

,“/

,’,

.,, ,:,:,.:

x

prototype

. .

. . “ . /“

.,, ., .,, , .:.:.,

,.:/

iii

a video-based

that provides a large area shared drawing space between remote sites. A schematic of a VideoWhiteboard system between two sites is shown in Figure 5.

... ...; .

i

‘.

\.

FOR SHARED DRAWING

A TOOL

P ‘;’‘ ‘

LL?;

while

In addition design of

,,, ., .,., ,.. :::::.

‘N ‘“:,..



//

marks.

display

the marks needed to

Whiteboard.

display

video camera

}1

of

avoiding blocking the overhead camera with one’s head is uncomfortable and compounded the parallax

a different sense of cursors interacting in a

screens (20” diagonal) restricted the amount of text and graphics that could be drawn before effectively Parallax sometimes made it filling the screen.

\

practice,

made by the users because

Users could only erase own screen and sometimes

VIDEOWHITEBOARD

0

work

sketchpad.

We also observed VideoDraw.

observing

prototypes

the glass thickness between the phosphor layer of the display (where others’ marks appeared) and the glass surface of the display (where a user’s marks

us explore new ways of providing a sense of copresence among remote collaborators. The video image of the users’ hands working together over the drawing presence

drawing

activity,

even occasionally in the same place

same time, an interaction plished when working

collaborative

that

i

between

two sites

“.;: .:,. ... ::,..

(

al

mode,

Each site is equipped with a wall-mounted rear projection screen (approximately 4.5’ x 6’), a video camera,

a video

projector,

and

appropriate

even

The

side of the screen

However,

of

the

projector

entire

the user,

screen

at the other

and

captures

sends

site, which

it

to

presents

user’s

video

each

other.

Users

can

projecting

onto the

write,

used

in

there

point

in VideoWhiteboard

is in some

to the computer-generated VIDEOPLACE is a fundamental

of view.

silhouette

[Krueger,

1982].

difference

In VideoWhiteboard,

from

the

the input

screen for drawing marks and casting shadows is the same as the output screen for projecting the remote collaborators’ marks and shadows. Thus,

the image

users can add marks and gestures directly over the marks made by their remote collaborators. In the drawing applications of VIDEOPLACE, the input focus (drawing in space) and the output focus (watching a computer monitor) are separated, adding

of their remote collaborators’ gestures and actions. An audio connection also enables the collaborators with

effect

similar

image

an image the

shadowy

ways

on that screen. As each user draws on the screen, those marks are imaged by the camera and projected Along with an onto the screen at the other site. image of the marks, the camera also transmits a shadow of the collaborator to the screen at the remote site. The collaborators see a composite image of real and “video” marks, as well as shadows

to talk

it is actually

audio

equipment. Users draw on the smooth front surface of the screen using standard dry erase whiteboard markers. The video camera, located on the opposite from

though

rear of the screen.

a level

collaborators’

of indirection marks

between

the

remote

and actions.

draw,

erase, and gesture at the VideoWhiteboard screens much as if they were working together at a shared whiteboard. Figure 6 shows a user interacting with

AND VIDEODRAW COMPARING VIDEOWHITEBOARD Like VideoDraw, VideoWhiteboard allows each user to share a drawing space and naturally draw,

a remote

gesture, and interact in that drawing space. Each user has a common view of the drawing space and

collaborator

through

a VideoWhiteboard.

Users at each site share a correct orientation to the and “left” have appropriate display surface— “right” meanings to the collaborators at both sites. Since

can make meaningful diectic references (e. g., “this one”, “here”) to objects or locations in the drawing

the camera

sketches created by a collaborator at the remote site. They can gesture over sketches and the remote

is on the opposite

side of the screen

space.

from

the user, it actually captures an image that is leftright reversed. This mirror image is corrected by operating the video projector in front projection

Figure

6: Interacting

with

a remote

can directly

augment

collaborators

can see those

the

sketch.

referent

collaborator

318

Users

through

Users

and interact

gestures between

VideoWhiteboard

in relation remote

over

to sites

even

have

such

that

concurrent more

than

access to the one person

drawing

space,

can be working

of people remotely collaborating for about one hour. Although we have not completed an extensive analysis, our initial observations have raised some interesting issues.

in

the same area at the same time. Figure 7 shows a composite image of the shadows of two collaborators from different sites superimposed on each other to show how they same space.

are working

closely

together

Features

in the

in Using Videowhiteboard

Collaborating

through

the familiar whiteboard. used when pointing both

VideoWhiteboard

builds

experience of working Most of the mechanisms working

together

and referring

hands

or multiple

over a whiteboard

to drawings, fingers,

on

together at for interaction

(e.g.,

gesturing

body

a

with

language

for

eliciting responses or demonstrating reaction) appear to work among remote collaborators using VideoWhiteboard. Like using a whiteboard, VideoWhiteboard allows several people to be working on the drawing

screen

collaborators to work screen

at the same time.

who are working

closely without

in

the

having

same their

other’s way—more closely than they were physically working whiteboard. Figure

7: A composite

collaborators

image

can work

showing

together

user

to work together VideoWhiteboard

shows

collaborators’ a full

color

image

whereas

of their

remote shadow screen

and

the

users

are

gestures

positioned

sides of the screen, obviating VideoDraw where the user’s camera’s

view

of the

also less of a problem parallax having

problem to view

drawing

(compare

on opposite

in VideoDraw

surface.

Parallax

in

each

seems to convince

some

users

that

they

was off to one side (not

on or behind

is

since the

was aggravated

surface

get

would be possible if around the same

location. The visual effect of seeing the of the remote collaborator projected onto the

the audio

one problem with head may block the

in VideoWhiteboard,

the drawing

bodies

sites

drawing

off axis from

by the

overhead camera. In this manner, VideoWhiteboard overcomes some of the limitations that were encountered in using Video Draw. OBSERVING VIDEOWHITEBOARD As with all of the prototype

IN USE

shared

drawing

tools

being developed at PARC, we observed people using VideoWhiteboard in realistic work activity. Videowas set up connecting two rooms in Whiteboard different sections of the building. A half-duplex audio connection was provided by speakerphones. In addition to several short, informal uses of VideoWhiteboard,

we observed

two sessions

are

talking to their collaborator through a translucent sheet of glass. In one observed incident, a user did not hear her remote collaborator clearly and cocked her ear toward the projection screen while asking him to repeat what he said, as shown in Figure 8. Even though the speakerphone that was producing

Video Draw

Figures 3 and 6). VideoWhiteboard conveys gestures of the upper body, while VideoDraw shows mostly hand gestures. In VideoWhiteboard, the camera

remote

of the

A common initial reaction to using VideoWhiteboard is that it feels like the remote collaborator is located on the other side of the screen, instead of in a

around the screen within a shows a shadow of the gestures,

area

how remote

in the same place

Unlike VideoDraw, VideoWhiteboard has a large screen surface, which offers the users more drawing area. The large screen also allows more than one site. remote

It also allows

between

Figure projection

of pairs

319

8: A user directs her ear toward the screen as part of a gesture to hear her remote collaborator better

the

screen), response

her gesture from her

helped remote

enunciated

his comment.

Although

this

impression

evoke the desired collaborator; he

of

interacting

with

someone on the opposite side of a translucent screen is false, this illusion seldom disrupts (and may even help) the users’ ability to interact with each other in this situation. Even if the users are not operating under evoke

that impression, VideoWhiteboard appropriate mechanisms for

through

it.

Users

quickly

view of the marks collaborators, and their

remote interesting

the

shadows

superimposed

that

they

share

a

on the screen with their remote they can see the shadows that

collaborators

An

realize

appears to interacting

feature of

cast on the screen. of VideoWhiteboard

the

remote

direct ly on the

collaborators screen

Figure

is that

image

shadows

are

Limitations Several

collaborators

at the drawing collaborators.

alternated

between

attending to the collaborators’ drawing surface at the same time. In one sense, VideoWhiteboard VideoDraw’s ability to convey because entire

large

VideoWhiteboard conveys gestures body. VideoWhiteboard can that

involve

sustain The

troublesome

gesture

an example of ‘