VIDEOWHITEBOARD: VIDEO SHADOWS TO SUPPORT REMOTE COLLABORATION John Scott System Xerox
C. Tang* L. Minneman
Sciences
Palo 3333 Palo
Alto
Laboratory Research
Center
Coyote Hill Road Alto, CA 94304
[email protected],
(415)494-4359
[email protected],
(415)494-4353
ABSTRACT
VideoWhiteboard remote shared
is a prototype drawing activity.
whiteboard-sized orators
who
each user
shared
in remote
of collaborators
development empirical
of studies
including shared
sites.
at the
remote
VideoWhiteboard
is
experiences
in
support
It allows
and a shadow
of collaborative
drawing
to
It provides a space for collab-
drawing
are located
to see the drawings
gestures
tool
The
based
drawing
using
prototype.
of the
site.
the
on
activity, VideoDraw
VideoWhiteboard
ables remote collaborators to work if they were sharing a whiteboard,
en-
together much as and in some ways
allows them to work together even more closely if they were in the same room.
than
Figure
1: Scene from
Sharing KEYWORDS:
gesture,
systems,
collaborative
video,
user interface,
shared
design
drawing,
process.
a common
resource
needed
nication
between
remote INTRODUCTION
Over
two
thousand
began
entertaining
plays
[March,
brightly
that
ago,
the imperial
the rear
shadows
years
1938].
colored
against
flat
This
court form
puppets
surface
Chinese with
of
that
of a backlit
in
artisans
are screen,
can be seen by the audience
Over
casting
in front
puppets
are
An Indonesian
granted
to
provided
copy that
without
fee
the copies
all or part are not made
of this
et
of
commercial
advantage,
other.
design
who
This
project
commu-
are
need
in a study
physically
for
a shared
of collaborators
that
sites [Olson
or distributed
last
decade,
that
partially
[1979]
collaborators.
1. to
was
& Bly,
distributed in press].
reported
blackboard
several
systems
address on for
the
this
have
need.
development
teleconferencing.
been
O’Boyle of an Many
Collaborators
who
are separated
by
geographical distance tell tales of sending faxes back and forth while talking on the phone in order to have timely interaction over graphical Shared window systems that enable information. people to interact over a common view of text and
is for
notice and the title of the publication and its data appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, raquiras a fse and/or specific permission. e 1991 ACM 0-89791 -383 -3/91 /0004 /0315 . ..$1.50 direct
each
is an important
graphical
current video teleconferencing facilities include overhead cameras or video copy stands for presenting images of drawings to remote
barely
material
al.
the
electronic
variety
of this art form, wayang kdit, is shown in Figure VideoWhiteboard uses a similar shadowy effect help remote collaborators work together. Permission
space
interactive
collaborators
two remote
developed
the
manipulate
to the audience.
play performance
uses
are
to
a software
pressed
project distinct shadows shadows of the rods that
used
drawing
for
space was noted
between
shadow
drama
the screen. The puppets onto the screen while the perceptible
from
drawing
a shadow
the ACM copyright
“Current address of first author is: John Tang, Sun 2550 Garcia Avenue, Mountain Microsystems, 94043,
[email protected], View, California (415)336-1636. 315
graphics
through
a computer
network
become review).
available These
(see [Lauwers approaches,
& Lantz, however,
clumsy or limited in their interaction over drawings familiar
with
have
ability to support fluent in a way that people are
from face-to-face
collaboration.
VideoWhiteboard is one of a series of prototype that support collaborative drawing activity natural
and familiar
also
19901 for a are often
way.
It uses video
tools in a
technology
to connect whiteboard-sized drawing surfaces It provides a shared between remote locations. “virtual whiteboard” that allows collaborators to interactively others’
create
gestures
marks
in relation
and see shadows
of each
to those
Remote
marks.
Figure
2: Schematic
Figure
3: User’s
of 2-person
VideoDraw
collaborators can draw on, erase, and gesture at the VideoWhiteboard screens much as if they were The interacting around a shared whiteboard. development empirical [Tatar,
of VideoWhiteboard
is based
studies of collaborative 1989] and is closely
VideoDraw
prototype
[Tang&
drawing related
Minneman,
This paper describes VideoWhiteboard process by which it is being developed. with
a review
of VideoDraw
and
describe
on our activity to the
1990]. and the We begin how
our
experiences with VideoDraw led to the development of VideoWhiteboard. Then we describe what VideoWhiteboard is and report on early observations of Finally, we discuss some issues people using it. raised
in these preliminary
of VideoWhiteboard, prototype and about in general.
observations
both about collaborative
the design of the drawing activity
in Figure
2.
It consists
each other’s marks and the hand made in relation to those marks. view of a VideoDraw
collaborative representation
is shown
in Figure
screen
drawing prototypes. An of this development process
4. An interdisciplinary
working
videotape records of face-to-face collaborative [Tang et al., 1990]. The analysis focused on
drawing activity, from this analysis
of an
Observing
interconnection of cameras aimed at video display screens. Users draw on the video display screen (using dry erase whiteboard markers) and those marks and accompanying hand gestures are imaged by the camera and displayed on the other screen. This arrangement creates a composite shared drawing surface where the collaborators can see
typical
of other idealized
study work
[Tang& Minneman, 19901 is a prototype enables collaborators to share a video A schematic diagram of a 2-person is shown
of a VideoDraw
group of anthropologists, computer scientists, and designers applied interaction analysis methods to
EXPERIENCES WITH VIDEODRAW VideoDraw tool that sketchpad. VideoDraw
view
of the use
how
and a subset of the observations led to the design of VideoDraw. people
used
VideoDraw
and
com-
paring it to how people work face-to-face led to the development of VideoWhiteboard. Although this idealization implies a linear sequence of steps, the actual process involved alternating between looking at various kinds of collaborative work (face-to-face, using VideoDraw, using other shared drawing tools) and modifying the designs of VideoDraw and VideoWhiteboard.
gestures that are Figure 3 shows a
screen. Our observations of people using confirmed that they often used hand
While the design of VideoDraw was informed by studies of collaborative drawing activity, studying VideoDraw in use also contributed to a better understanding of that activity and the development
VideoDraw gestures to
enact simulations or mediate their interaction, and that these gestures were often made with respect to a referent sketch in the drawing space. Using
316
Figure
4: Iterative
development
of tools to support analyzing
VideoDraw,
remote
collaborators
were
the process of creating and referring (rather than just seeing the resulting which
is an important
activity.
resource
Collaborators to be drawing
VideoDraw
drawing
surface.
and building
able
to see
difficult
to drawings drawings),
in shared
which cannot together over
Furthermore,
drawing used at the
surface provides than, for example,
computational
to align
were drawn). made on their
be accoma single
VideoDraw
through
marks
helped
several
The
limitations
relatively
in the
small
video
request
others
to erase their
upward
facing
CRT
viewing ifications
problem. to the
observations
informed
\ .. \
VideoWhiteboard
use of
Straddling at the same
to suggesting VideoDraw, of design
the
an time
modthese Video-
is
,’,
/“
,. :,:. .n .. .: .:,: :.:.:. .,.,.:
~
J
t
video projector
Figure
\,’.
:.:,:.
II ,.”
..,,, .....
.
\,’. 1. .
,: .:.:
I
5: Schematic
Ii,: .:..
tool
k ....: .::.. ,,
l-l ....:. ...... ,.,.,. .:.:.:
317
.,
x,
::::.: .:,.,. .:. :.:,,,
system
. =
;:.:.:.j
of VideoWhiteboard
o
.
/“
.
...... .:.:.,
/’
1
. .
..:. :.:. ., ....
/
,“/
,’,
.,, ,:,:,.:
x
prototype
. .
. . “ . /“
.,, ., .,, , .:.:.,
,.:/
iii
a video-based
that provides a large area shared drawing space between remote sites. A schematic of a VideoWhiteboard system between two sites is shown in Figure 5.
... ...; .
i
‘.
\.
FOR SHARED DRAWING
A TOOL
P ‘;’‘ ‘
LL?;
while
In addition design of
,,, ., .,., ,.. :::::.
‘N ‘“:,..
‘
//
marks.
display
the marks needed to
Whiteboard.
display
video camera
}1
of
avoiding blocking the overhead camera with one’s head is uncomfortable and compounded the parallax
a different sense of cursors interacting in a
screens (20” diagonal) restricted the amount of text and graphics that could be drawn before effectively Parallax sometimes made it filling the screen.
\
practice,
made by the users because
Users could only erase own screen and sometimes
VIDEOWHITEBOARD
0
work
sketchpad.
We also observed VideoDraw.
observing
prototypes
the glass thickness between the phosphor layer of the display (where others’ marks appeared) and the glass surface of the display (where a user’s marks
us explore new ways of providing a sense of copresence among remote collaborators. The video image of the users’ hands working together over the drawing presence
drawing
activity,
even occasionally in the same place
same time, an interaction plished when working
collaborative
that
i
between
two sites
“.;: .:,. ... ::,..
(
al
mode,
Each site is equipped with a wall-mounted rear projection screen (approximately 4.5’ x 6’), a video camera,
a video
projector,
and
appropriate
even
The
side of the screen
However,
of
the
projector
entire
the user,
screen
at the other
and
captures
sends
site, which
it
to
presents
user’s
video
each
other.
Users
can
projecting
onto the
write,
used
in
there
point
in VideoWhiteboard
is in some
to the computer-generated VIDEOPLACE is a fundamental
of view.
silhouette
[Krueger,
1982].
difference
In VideoWhiteboard,
from
the
the input
screen for drawing marks and casting shadows is the same as the output screen for projecting the remote collaborators’ marks and shadows. Thus,
the image
users can add marks and gestures directly over the marks made by their remote collaborators. In the drawing applications of VIDEOPLACE, the input focus (drawing in space) and the output focus (watching a computer monitor) are separated, adding
of their remote collaborators’ gestures and actions. An audio connection also enables the collaborators with
effect
similar
image
an image the
shadowy
ways
on that screen. As each user draws on the screen, those marks are imaged by the camera and projected Along with an onto the screen at the other site. image of the marks, the camera also transmits a shadow of the collaborator to the screen at the remote site. The collaborators see a composite image of real and “video” marks, as well as shadows
to talk
it is actually
audio
equipment. Users draw on the smooth front surface of the screen using standard dry erase whiteboard markers. The video camera, located on the opposite from
though
rear of the screen.
a level
collaborators’
of indirection marks
between
the
remote
and actions.
draw,
erase, and gesture at the VideoWhiteboard screens much as if they were working together at a shared whiteboard. Figure 6 shows a user interacting with
AND VIDEODRAW COMPARING VIDEOWHITEBOARD Like VideoDraw, VideoWhiteboard allows each user to share a drawing space and naturally draw,
a remote
gesture, and interact in that drawing space. Each user has a common view of the drawing space and
collaborator
through
a VideoWhiteboard.
Users at each site share a correct orientation to the and “left” have appropriate display surface— “right” meanings to the collaborators at both sites. Since
can make meaningful diectic references (e. g., “this one”, “here”) to objects or locations in the drawing
the camera
sketches created by a collaborator at the remote site. They can gesture over sketches and the remote
is on the opposite
side of the screen
space.
from
the user, it actually captures an image that is leftright reversed. This mirror image is corrected by operating the video projector in front projection
Figure
6: Interacting
with
a remote
can directly
augment
collaborators
can see those
the
sketch.
referent
collaborator
318
Users
through
Users
and interact
gestures between
VideoWhiteboard
in relation remote
over
to sites
even
have
such
that
concurrent more
than
access to the one person
drawing
space,
can be working
of people remotely collaborating for about one hour. Although we have not completed an extensive analysis, our initial observations have raised some interesting issues.
in
the same area at the same time. Figure 7 shows a composite image of the shadows of two collaborators from different sites superimposed on each other to show how they same space.
are working
closely
together
Features
in the
in Using Videowhiteboard
Collaborating
through
the familiar whiteboard. used when pointing both
VideoWhiteboard
builds
experience of working Most of the mechanisms working
together
and referring
hands
or multiple
over a whiteboard
to drawings, fingers,
on
together at for interaction
(e.g.,
gesturing
body
a
with
language
for
eliciting responses or demonstrating reaction) appear to work among remote collaborators using VideoWhiteboard. Like using a whiteboard, VideoWhiteboard allows several people to be working on the drawing
screen
collaborators to work screen
at the same time.
who are working
closely without
in
the
having
same their
other’s way—more closely than they were physically working whiteboard. Figure
7: A composite
collaborators
image
can work
showing
together
user
to work together VideoWhiteboard
shows
collaborators’ a full
color
image
whereas
of their
remote shadow screen
and
the
users
are
gestures
positioned
sides of the screen, obviating VideoDraw where the user’s camera’s
view
of the
also less of a problem parallax having
problem to view
drawing
(compare
on opposite
in VideoDraw
surface.
Parallax
in
each
seems to convince
some
users
that
they
was off to one side (not
on or behind
is
since the
was aggravated
surface
get
would be possible if around the same
location. The visual effect of seeing the of the remote collaborator projected onto the
the audio
one problem with head may block the
in VideoWhiteboard,
the drawing
bodies
sites
drawing
off axis from
by the
overhead camera. In this manner, VideoWhiteboard overcomes some of the limitations that were encountered in using Video Draw. OBSERVING VIDEOWHITEBOARD As with all of the prototype
IN USE
shared
drawing
tools
being developed at PARC, we observed people using VideoWhiteboard in realistic work activity. Videowas set up connecting two rooms in Whiteboard different sections of the building. A half-duplex audio connection was provided by speakerphones. In addition to several short, informal uses of VideoWhiteboard,
we observed
two sessions
are
talking to their collaborator through a translucent sheet of glass. In one observed incident, a user did not hear her remote collaborator clearly and cocked her ear toward the projection screen while asking him to repeat what he said, as shown in Figure 8. Even though the speakerphone that was producing
Video Draw
Figures 3 and 6). VideoWhiteboard conveys gestures of the upper body, while VideoDraw shows mostly hand gestures. In VideoWhiteboard, the camera
remote
of the
A common initial reaction to using VideoWhiteboard is that it feels like the remote collaborator is located on the other side of the screen, instead of in a
around the screen within a shows a shadow of the gestures,
area
how remote
in the same place
Unlike VideoDraw, VideoWhiteboard has a large screen surface, which offers the users more drawing area. The large screen also allows more than one site. remote
It also allows
between
Figure projection
of pairs
319
8: A user directs her ear toward the screen as part of a gesture to hear her remote collaborator better
the
screen), response
her gesture from her
helped remote
enunciated
his comment.
Although
this
impression
evoke the desired collaborator; he
of
interacting
with
someone on the opposite side of a translucent screen is false, this illusion seldom disrupts (and may even help) the users’ ability to interact with each other in this situation. Even if the users are not operating under evoke
that impression, VideoWhiteboard appropriate mechanisms for
through
it.
Users
quickly
view of the marks collaborators, and their
remote interesting
the
shadows
superimposed
that
they
share
a
on the screen with their remote they can see the shadows that
collaborators
An
realize
appears to interacting
feature of
cast on the screen. of VideoWhiteboard
the
remote
direct ly on the
collaborators screen
Figure
is that
image
shadows
are
Limitations Several
collaborators
at the drawing collaborators.
alternated
between
attending to the collaborators’ drawing surface at the same time. In one sense, VideoWhiteboard VideoDraw’s ability to convey because entire
large
VideoWhiteboard conveys gestures body. VideoWhiteboard can that
involve
sustain The
troublesome
gesture
an example of ‘