Editor: Frank Nack Center for Mathematics and Computer Science (CWI)
Media Impact Pictures at an Exhibition Frank Nack CWI
I
’m a museum fan. Besides the art, it’s the museum’s intrinsic ambivalence of telling stories about the real world in an environment that couldn’t be further removed that fascinates me. Museums encourage discourse—I had the best conversations about art and other topics in museums with total strangers. Above all, though, they stimulate creative urges by working on primary sensory material—such as sound, color, or light— reaching for the deepest layers of the emotional memory. Just a few days ago while strolling round the Rijksmuseum in Amsterdam, Judith Leyster’s painting Serenade caught my eye. The painting portrays a lavishly dressed young man singing a serenade and playing the lute. The young singer seems entirely engrossed in his music and out of the blue I started to whistle the second movement of Modest Mussorgsky’s suite, Pictures at an Exhibition, which represents the painting of a troubadour by the walls of a ruined medieval castle. How nice it would be, I thought, if there was music composed for Leyster’s painting, and an information system could let me know right now. My mind drifted away, envisaging the impact of information technology for museum environments.
The museum paradigm The primary aim of museums is to collect, preserve, and support the study of historical, artistic, or scientific artifacts. A collection exhibition is the result of that work, but the exhibition should also garner income to guarantee the museum’s continuation. Generally the biggest challenge of exhibits has to do with limited space. Thus, mere fractions of the collection can be presented at any one time. Consequently, museums tend to display the more popular artifacts, which then can also be found as replicas, reprints, or motifs on various goods in the associated shop. This sort of art merchandising covers essential parts of the budget, at least for world famous collections that can guarantee a constant stream of visitors.
14
1070-986X/02/$17.00 © 2002 IEEE
For most institutions, the dilemma is that today’s visitors no longer desire the traditional canon that supplies them with similar material in comparable presentation formats. On the contrary, the audience wishes to be confronted with connection, context, and uncommon perspectives. The strategy within the museum community is clear—feature more in-house supershows and presentations of potentially interesting artifacts on the Web. Thus, the museum has turned into entertainment. The successful executor of this policy is the Salomon-Guggenheim Foundation. The foundation has a culture trust built on profitable joint ventures with other large collections (such as the Hermitage in St. Petersburg and the Kunsthistorische Museum in Vienna), establishing itself as an art stock house. The foundation owns impressive architectural branches at tourist-relevant locations where superexhibitions find their clients. This, however, is the interesting step—Guggenheim established a dotcom environment. At the moment, the site (http://www.guggenheim.com) offers a sophisticated, visually pleasing, fun, and cost-free environment providing valuable and detailed information about current exhibitions and stimulating advertisements for exhibitions to come. The plans for the future target services that appeal to a section of the market willing to pay for the click beyond introductory information. Guggenheim’s gigantic image database (containing an estimated 65,000 drawings, 1 million prints, and thousands of photos from artifacts) provides the means for a global business with digital reproductions—allowing the purchase of single images or the exclusive lease of a cluster of images. The lease is of particular interest for companies that want to enhance a special event (such as a company event or the shareholders’ meeting) with large liquid crystal display (LCD) screens showing masterpieces that won’t be exhibited anywhere else during the reserved period of time. Other planned services cover vir-
Figure 1. Expositur—a virtual knowledge space by Mathias Fuchs and Sylvia Eckermann.
tual access to current exhibitions, marketing of seminars lined with art history, provisions of a tourist department arranging individual cultural programs, or even booking a comprehensive cultural trip for the Guggenheim.com member (including reservation of flights, hotels, tickets for museums, opera or theater performances, and so forth).
about the space to find out facts. Naturally, the virtual media architecture mustn’t be 3D only because the physical space proposes this analogy. The important step is to understand that the virtual and the physical space represent two sides of the same coin—the domain semantics.
House of ideas Physical and virtual spaces
Our experience-hungry society will mobilize the latent capabilities of museums to communicate knowledge in a compelling and engaging way—letting a visitor experience an event as a continuous, flexible, and networked exchange of ideas. In the physical building, Webpads or palms provided by the museum facilitate the individual access to the ongoing event as well as additional knowledge from the virtual space. This information might lead to another piece of art in the archive or to a different exhibition. The presentation might be communicated in the form of verbal, gestical, musical, iconic, graphic, or sculptural expressions, where the adaptive level of discourse for members of the institution is less sketchy, because the system holds information about the individual preferences on topics and presentation techniques. Visitors can place comments on artifacts or their presentation. Additional screens at various places in the museum make selected requests and knowledge publicly available and provide ways to make others aware of available sources. Individuals accessing the museum virtually can switch to an exhibition and explore the action in the various rooms, investigate artifacts, provide comments, and contact other visitors if they wish. The museum is an individual-supporting—but still communal—space, facilitating visitors as they absorb knowledge.
April–June 2002
While the Guggenheim approach is more visionary than most other schemes, it’s still captivated in the concept of the house of memories, where the virtual wing merely mirrors the physical. This needn’t be the case. Thomas Fuchs and Sylvia Eckermann, two media artists from Vienna, demonstrate with their work on how to link a public space with private experience by setting up virtual audio–visual units.1 In collaboration with 10 Austrian museums, they developed a virtual space built on Epic Games’ Unreal Engine. They made the content accessible and comprehensible to users of different ages, educational backgrounds, and levels of computer literacy. The work has two interesting aspects about it. First, the provided content in the virtual museum uses material that usually isn’t exhibited by the associated museums (such as artificial limbs) or that can’t be displayed because only descriptions and no real artifacts are available (such as an ancient fish once populating the Danube). Second, Fuchs and Eckermann developed a system of connotations among the artifacts to be exhibited, which they then translated into a spatial structure of rooms, corridors, and places of different size, shape, remoteness, or proximity. Figure 1 shows an example of this. (For other exhibition images, visit http://www.t0.or.at/ ~fuchs-eckermann/expositur_fr.htm.) What the visitors really explore is a semantic structure by navigating virtual spaces with the topics contained in the rooms. The semantic structures facilitate a visualization that forces users to move
The problem zone The scenario from the previous section makes me frown, but it’s not the computational com-
15
Media Impact
Web Resources Here’s a list of museum-related sites that might be interesting to you: ❚ Albertina, Vienna: http://www.albertina.at/index_d_fl.html ❚ Getty Museum Los Angeles: http://www.getty.edu/ ❚ Guggenheim.com: http://www.guggenheim.com/index2.html ❚ Guggenheim Museum, New York: http://www.guggenheim.org ❚ Hermitage Museum, St. Petersburg: http://www.hermitagemuseum.org ❚ Kunsthistorische Museum, Vienna: http://www.khm.at/system2.html?/ static/page1.html ❚ Louvre, Paris: http://www.louvre.fr/ ❚ National Gallery of Art, Washington D.C.: http://www.nga.gov/ ❚ Rijksmuseum, Amsterdam: http://www.rijksmuseum.nl ❚ Victoria and Albert Museum, London: http://www.vam.ac.uk/
IEEE MultiMedia
plexity of the hardware or network that worries me. Museums are controllable environments, and I’m sure that the results produced by our research community can essentially cope with the challenges provided by communal spaces. Moreover, the tremendous performance of the game and digital entertainment industry with respect to multiplayer environments confirms this.2 It’s the authoring aspect that sends shivers down my spine. Not only do we require tools that support the design and realization of complex exhibitions—a field already full of difficult tasks—but we must also provide database networks to supply the rich material that lets us establish persuasive contexts. These repositories will be a great aid during the production phase of an exhibition if we can additionally provide systems for authoring media. The systems will let designers and domain experts use their creativity in familiar ways as well as their human processing to extract the significant syntactic, semantic, and semiotic aspects of the media’s content during production or any time after. The result will be semiautomatically generated machine-readable descriptions based on a formal language, such as the Resource Description Framework (RDF) Schema or the DARPA Agent Markup Language and Ontology Layer Interface
16
(DAML+OIL), facilitating the use of media-based information for diverse purposes. Then, the repositories also form an essential part of an exhibition. Given the unpredictable nature of the relationship between visitor and presentation in a dynamic environment, it’s infeasible to create all relevant media documents for a virtual space in advance. We need automated processes for generating visually pleasing media documents. The tools for designing and maintaining such processes are in their infancy. Their development depends on our better understanding of objective measurements for media units (such as color, shape, or texture), representing prototypical style elements and how they can be combined with high-level conceptual descriptions supporting contextual and presentational requirements. Research in industry and academia has let us characterize audio–visual information on a conceptual and a perceptual level by using objective measurements based on image or sound processing, pattern recognition, and so forth. However, the object of our desire was always the final media product. It lies in front of us, complete and ready for the vivisection. Yet, the future challenge is not the analysis but the composition. Moreover, I wonder if we’ve ignored potential users of our research for too long and maneuvered our work and ideas into an unfavorable position. Looking at the two main conferences in our field, the IEEE International Conference on Multimedia and Expo and the ACM Conference on Multimedia (ACM MM), hardly any traces of media producers or large content providers exist. At the 1998 ACM MM in Bristol, a number of media artists were invited. The 2001 ACM MM featured a session for media production (see http:// www.acm.org/sigs/sigmm/MM2001/ep/toc.html# Wp1). Occasionally, there might be a workshop (see, for example, http://www.acm.org/sigs/ sigmm/MM2000/ep/toc.html#GAP). Even MPEG7, which is supposed to address relevant problems, just manages to appear in demo sessions or gets discussed during breaks.
Tick-tock says the clock It took nearly 30 years of steady infiltration of technological advances to establish media authoring tools (such as Macromedia’s Director/Shockwave, Flash, or Dreamweaver) that let us communicate ideas in a visually pleasing and entertaining way. The Web manifests these tools’ success, and most museums rely on this type of
technology to create their animated material. However, when it comes to retrieving information from these animations, the animations behave like black holes where the semantics of the content is lost in a proprietary stream of bites. If we really want to achieve dynamic and personalized environments as required for compelling and engaging experiences of rich multimedia data, and if we want the revolution of the semantic media Web, we have to collaborate with those who create it. And there’s still time to do this, although most likely not 30 years. The Web actually forces us not only to rethink paradigms but also how content providers use multimedia. Most of them have to make their catalogs and media units suitable for the Web. This often means transforming databases and re-representing existing ontologies, such as Getty’s Art and Architecture Thesaurus, the Library of Congress’ Thesaurus of Graphic Materials, the Devel-
opments in International Museum and Cultural Heritage Information Standards. If we want to be revolutionary researchers, we must make the revolution irresistible. If we can’t, then the pictures at an exhibition are just a dream within a dream. MM
References 1. T. Fuchs and S. Eckermann, “From ‘First-Person Shooter’ to Multi-User Knowledge Spaces,” Proc. First Conf. Computational Semiotics for Games and New Media (COSIGN 2001), 2001, pp 83-87, http://www.cwi.nl/conferences/Cosign2001/. 2. F. Nack, “Play the Game,” IEEE MultiMedia, vol. 8, no. 1, Jan./Mar. 2001, pp. 8-10. Contact Media Impact editor Frank Nack at CWI, Kruislaan 413, PO Box 94079, 1090 GB Amsterdam, The Netherlands, email
[email protected].
COMPUTER
PURPOSE The IEEE Computer Society is the world’s
MEMBERSHIP Members receive the monthly magazine COMPUTER , discounts, and opportunities to serve (all activities are led by volunteer members). Membership is open to all IEEE members, affiliate society members, and others interested in the computer field.
BOARD OF GOVERNORS Term Expiring 2002: Mark Grant, Gene F. Hoffnagle, Karl Reed, Kathleen M. Swigger, Ronald Waxman, Michael R. Williams, Akihiko Yamada Term Expiring 2003: Fiorenza C. AlbertHoward, Manfred Broy, Alan Clements, Richard A. Kemmerer, Susan A. Mengel, James W. Moore, Christina M. Schober Term Expiring 2004: Jean M. Bacon, Ricardo Baeza-Yates, Deborah M. Cooper, George V. Cybenko, Wolfgang K. Giloi, Haruhisha Ichikawa, Thomas W. Williams Next Board Meeting:10 May 02, Portland OR
IEEE OFFICERS President: RAYMOND D. FINDLAY President-Elect: MICHAEL S. ADLER Past President: JOEL B. SYNDER
EXECUTIVE COMMITTEE President: WILLIS K. KING* University of Houston Dept. of Comp. Science 501 PGH Houston, TX 77204-3010 Phone: +1 713 743 3349 Fax: +1 713 743 3335
[email protected] President-Elect: STEPHEN L. DIAMOND* Past President: BENJAMIN W. WAH* VP, Educational Activities: CARL K. CHANG * VP, Conferences and Tutorials: GERALD L. ENGEL* VP, Chapters Activities: JAMES H. CROSS VP, Publications: RANGACHAR KASTURI VP, Standards Activities: LOWELL G. JOHNSON †
†
(2ND VP)*
VP, Technical Activities: DEBORAH K. SCHERRER(1ST VP)*
Secretary: HUGO M. FERNANDEZ VERSTAGEN
Secretary: DEBORAH M. COOPER* Treasurer: WOLFGANG K. GILOI* 2001–2002 IEEE Division VIII Director:
Treasurer: DALE C. CASTON
THOMAS W. WILLIAMS
VP, Educational Activities: LYLE D. FEISEL
2002–2003 IEEE Division V Director:
Executive Director: DANIEL J. SENESE
GUYLAINE M. POLLOCK
†
VP, Publications Activities: JAMES M. TIEN VP, Regional Activities: W. CLEON ANDERSON VP, Standards Association: BEN C. JOHNSON VP, Technical Activities: MICHAEL R. LIGHTNER President, IEEE-USA: LeEARL A. BRYANT
SOCIETY
WEB
SITE
The IEEE Computer Society’s Web site, at http://computer.org, offers information and samples from the society’s publications and conferences, as well as a broad range of information about technical committees, standards, student activities, and more.
largest association of computing professionals and is the leading provider of technical information in the field.
Executive Director: DAVID W. HENNAGE *voting member of the Board of Governors
†
COMPUTER SOCIETY O F F I C E S Headquarters Office 1730 Massachusetts Ave. NW Washington, DC 20036-1992 Phone: +1 202 371 0101 • Fax: +1 202 728 9614 E-mail:
[email protected] Publications Office 10662 Los Vaqueros Cir., PO Box 3014 Los Alamitos, CA 90720-1314 Phone:+1 714 821 8380 E-mail:
[email protected] Membership and Publication Orders: Phone: +1 800 272 6657 Fax: +1 714 821 4641 E-mail:
[email protected] European Office 13, Ave. de L’Aquilon B-1200 Brussels, Belgium Phone: +32 2 770 21 98 • Fax: +32 2 770 85 05 E-mail:
[email protected] Asia/Pacific Office Watanabe Building 1-4-2 Minami-Aoyama, Minato-ku, Tokyo 107-0062, Japan Phone: +81 3 3408 3118 • Fax: +81 3 3408 3553 E-mail:
[email protected]
EXECUTIVE STAFF Executive Director: DAVID W. HENNAGE Publisher: ANGELA BURGESS Assistant Publisher: DICK PRICE Director, Volunteer Services: ANNE MARIE KELLY Chief Financial Officer: VIOLET S. DOAN Director, Information Technology & Services: ROBERT CARE Manager, Research & Planning: JOHN C. KEATON 11-FEB-2002
17
