Digital and Paper Documents - ACM Digital Library

CHI 2006 · Work-in-Progress

April 22-27, 2006 · Montréal, Québec, Canada

PaperSpace: A System for Managing Digital and Paper Documents Jeff Smith

Sriram Subramanian

Department of Computer Science

Department of Computer Science

University of Saskatchewan

University of Saskatchewan

Saskatoon, SK, Canada

Saskatoon, SK, Canada

[email protected]

[email protected]

Jeremy Long Department of Computer Science University of Saskatchewan Saskatoon, SK, Canada

[email protected] Tanya Lung Department of Computer Science University of Saskatchewan Saskatoon, SK, Canada

Abstract Here we present PaperSpace a computer vision based document management system that allows users to combine paper and digital documents. Using PaperSpace users can locate paper copies of printed digital documents, retrieve digital versions of paper documents and fluidly move between digital and paper documents. The system works by tracking 2D identity and operation codes printed in the margins of each page of the document. Users can activate commands by selecting the command from the command bar at the bottom of each document. Through an informal evaluation we found that all users felt our system would be valuable in helping them organize their cluttered desk and manage digital and paper documents better.

[email protected]

Keywords Mohd M. Anwar Department of Computer Science

Document management system, Visual Codes, Tangible Interface, Paper Interface, Paperless Office

University of Saskatchewan Saskatoon, SK, Canada

ACM Classification Keywords

[email protected]

H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous.

Copyright is held by the author/owner(s).

Introduction

CHI 2006, April 22–27, 2006, Montreal, Quebec, Canada.

Despite the bold predictions in the early 1970s, the paperless office is still more myth than reality [2]. In fact, the use of paper documents within modern office

ACM 1-59593-298-4/06/0004.

1343



settings continues to rise, despite the presence of increasingly advanced digital alternatives. Recent investigations have expressed a growing need for an information management system that allows paper and digital documents to work in concert with the system making optimal use of both [2, 11]. We developed PaperSpace, a document management system that allows users to take advantage of the desirable aspects of digital documents and paper documents. The system manages academic papers using computer vision to track the current location of all paper documents printed by the user. PaperSpace aims to cut down redundancy in printed documents by informing users, before they print, if a hard copy of that document already exists nearby. Users can also use the system to locate misplaced paper or digital documents in their work area. In an informal evaluation of the system, we found that users generally liked it and felt that it would enhance their work environment.

the desk organization of a typical knowledge worker and noted that many have extremely cluttered desks and floors, filled with different piles. Yet these workers are seriously disrupted by changes made to this apparent “muddle”. Digital systems have proven notoriously poor at simulating the piling that occurs so frequently in office settings. Sellen and Harper [11] and Bondarenko and Janssen [2] suggested that future systems support users working in concert with both paper and digital documents. They believed that this could allow systems to effectively capture the advantages of both formats. The first approach to augmenting digital documents with the qualities of a paper document was the Digital Desk [12], where the virtual desktop was projected onto a tabletop surface to capture the qualities of a physical desk. Koike et al. have proposed an enhanced desk [6] that integrates paper and digital information on a desk and provides direct manipulation of the digital information with the hands and fingers of users.

Related Works With the concept of a paperless office still remaining a distant dream, many researchers have studied the organization of physical desks and the personal document management systems [5, 8]. Malone [8] identified two major units of desk organization – files and piles. Elements in files are explicitly titled and organized in some systematic order, sometimes with the file itself having a title. On the other hand, elements in piles are not explicitly titled and generally not arranged in any particular order. Most often the pile itself has no title. Kidd [5] analyzed

Arai et al. proposed PaperLink [1], a system that uses conventional paper for input, conventional (computer) displays for output. It creates hyperlinks between existing paper documents and the computer. There have been increasingly successful implementations of paper tracking using computer vision. Robertson and Robinson have proposed Live Paper [10] that uses a video camera and a data projector to turn sheets of paper into I/O devices. The Live Paper system provides audio and video augmentation, including collaboration with remote users through videowriting (real-time

1344



extraction of annotations.

writing

on

a

page)

and

projected

Using paper as an interface to a digital document allows user to move freely from the paper world to digital world. Liao and Hinckley’s PapierCraft [7] provides a command system that allows users to manipulate digital documents directly using their printouts. The system provides a digital pen to annotate the paper printout by copying, pasting, and linking an area within the document.

figure 1. Example papers printed with visual codes on each corner of the page and a command bar at the bottom of the page.

PaperSpace The PaperSpace system has been designed with the aim of allowing knowledge workers to share paper and digital versions of PDF documents. The main design goal of the PaperSpace system is to digitally augment paper to capture the advantages of both the physical and digital domains within one system. The system

consists of five modules: a database, a tracking code generator, a vision system, a tracker, and a user interface. The system is built around one central database that contains listings for any ACM documents that have been inserted into the PaperSpace system. We used documents from the ACM Library which are cataloged by Digital Object Identifier (DOI). This library was used because DOI numbers will permit us to develop a mechanism for downloading BibTex-formatted citation information to automatically populate the database records as new papers are added to the system. Our database also stores information on print jobs that have been initiated through the PaperSpace system. Users of the system will be able to browse this central database through the local PaperSpace user interface client. When the user prints a digital document through the system, special 2D bar-codes are created by the code generator and embedded in the margins of the printed document by the print module within the user interface client, creating a tagged paper document as shown in Figure 1. These tags are used primarily for document identification and command invocation. Document identification is accomplished by parsing the visual code that is attached to each corner of the document that encodes the following information: DOI, User ID, page corner identifier, Date Printed, Version and Page Number. These specially designed codes (see Figure 1 top pull out) are an extension to the common QR codes and have enough information to identify a paper without compromising minor spatial and rotational distortions.

1345



The PaperSpace system uses a medium resolution webcam to track the papers around the user’s desk. The vision system is also capable of capturing and parsing gestural operations performed on the command bar. This bar is a static image (see Figure 1) that contains useful commands such as annotate, link, email and more that can be performed on the document in question. To activate a command the user must cover both the activation button and the intended command as shown in Figure 2. We use command activation through an activation button to prevent accidental command invocations. The identification of both identification symbols and user commands is accomplished by the vision system. It performs an image segmentation to create a region-adjacency map of the image. In doing so, it can locate any visual codes or any properly obscured, and thereby invoked, commands. We chose the region-adjacency approach to minimize our sensitivity to camera orientation. At the moment, this is a fairly costly operation. We hope in future work to optimize our implementation, improving the vision system to real-time efficiency. Information regarding the identity and location of papers is passed from the vision system to a tracking module that runs on each local machine involved with PaperSpace. This tracking system parses the location data in order to build a coherent understanding of the paper’s location and orientation on the desktop. By comparing current and previous position of the paper the tracking module assembles the list of papers in a pile. In most situations, the tracker can maintain information on the position of a paper within a pile. This is useful for visualization purposes, and will allow for the tracking of all documents within ’sight’ of the PaperSpace system.

All the location and printing information is stored within one central PaperSpace database. Users can access this database through a query interface that allows them to search for previously printed documents. In this manner, they will be able to gain access both to a digital copy which can be reprinted, and the location of recently printed physical copies of that same document.

figure 2. Example of the command bar being used to give the annotate command to the system.

Early User Testing We did an informal evaluation of the system to solicit user feedback on its use and usefulness. Rather than show the system or let them use it, we showed users a 5 min video of people working with the system so that they could focus on the main aspects of the system rather than on the details of how the query interface looks or tracking speed of the camera. The video showed a user looking for a misplaced paper document and attempting to print a paper copy of a document his colleague recently printed. It then shows how using the query interface of the PaperSpace system the user can view an image of the physical desktop with the sought paper highlighted. It also shows how the query interface interrupts the user’s print request to inform them that a colleague recently printed the same document. We then explain how these features work and present our vision of the future where users can pile paper without worrying about misplacing documents or can foster collaboration with colleagues

1346



they didn’t realize were reading papers that interest them too. We interviewed 6 knowledge workers from an academic or IT office environment. They saw the video and then answered questions about its features, potential problems and value to current work practice. All interviewees were excited about the PaperSpace system. They particularly liked being able to track their paper copies. One participant stated “From my perspective, the system seems most useful as a means of organizer for a cluttered workspace”. Although most people liked the idea of the command bars on their paper documents, two of the participants were concerned about its size, visual effect and accidental invocations. Users were also concerned about privacy issues which we side-stepped in this early investigation of the system. The following comments sums up what most participants had in mind. One participant mentioned “I'm always looking for additional readings on my topics, so it would be useful to find the reading history of someone else who has interest in some of the same documents”. Another mentioned “Locating your copy is much more important (than credited for) because if you can’t find the copy you printed, you end up printing another”. Overall, the system was well received by the participants and they were excited with the features offered by the system.

Conclusion and Future Work In this paper we presented PaperSpace a document management system that allows users to manage digital and paper documents simultaneously. It provides a method for sharing, locating, filing and associating documents through either the paper or the digital copies. By providing a technique for dealing with these six needs, PaperSpace successfully extends the functionality of existing systems. Initial testing showed that users were positive and enthusiastic about usefulness and usability of the system. In the future we will look at deploying the system in an academic environment to solicit more detailed user feedback to confirm if users can indeed manage paper and digital documents better with the system. The system also raises privacy concerns that will need to be addressed. In future implementations, users will have the opportunity to turn off PaperSpace at different levels. They can direct the system to stop printing visual codes so the papers can be blind to the vision system or remove any document from the central database. We further plan to integrate the PaperSpace system with a networked printer so users can print documents through any application and use the query interface only when looking for a document.

References [1] Arai, T., Aust, D., and Hudson, S. (1997) PaperLink: a technique for hyperlinking from real paper to electronic content. In Proceedings of the SIGCHI conference on Human factors in computing system (Atlanta, Georgia, USA, March 22-27, 1997). CHI ’97, ACM Press.

1347



[2] Bondarenko, O. and Janssen, R. (2005) Documents at Hand: Learning from Paper to Improve Digital Technologies. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Portland, Oregon, USA, April 02 - 07, 2005). CHI '05. ACM Press. [3] Boardman, R. and Sasse, M. A. (2004) Stuff goes into the computer and doesn't come out: a cross-tool study of personal information management. In Proceedings of the SIGCHI conference on Human factors in computing systems (Vienna, Austria). Chi ’04, ACM Press. [4] Holman, D., Vertegaal, R., Altosaar, M., Troje, N., and Johns, D. (2005) Paper windows: interaction techniques for digital paper. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Portland, Oregon, USA, April 02 - 07, 2005). CHI '05. ACM Press. [5] Kidd, A. (1994) The marks are on the knowledge worker. Conference companion on Human factors in computing systems, (Boston, Massachusetts, United States, April 24-28, 1994). CHI’94, ACM Press. [6] Koike, H. (2001) Integrating paper and digital information on EnhancedDesk: a method for realtime finger tracking on an augmented desk system, ACM Transactions on Computer-Human Interaction (TOCHI) 8(4), December, 2001, Pages.307-322. [7] Liao, C., Guimbretiere, F. and Hinckley, K. (2005) PapierCraft: a command system for interactive paper. In Proceedings of the 18th Annual ACM Symposium on

User interface Software and Technology (Seattle, WA, USA, October 23 - 26, 2005). UIST '05. ACM Press. [8] Malone, T. W. (1983). How do people organize their desks?: Implications for the design of office information systems. ACM Trans. Inf. Syst. 1, 1 (Jan. 1983), 99112. [9] Pedersen, E. R., Sokoler, T., and Nelson, L. (2000) PaperButtons: expanding a tangible user interface. In Proceedings of the Conference on Designing interactive Systems: Processes, Practices, Methods, and Techniques (New York City, New York, United States, August 17 - 19, 2000). DIS '00. ACM Press. [10] Robertson, C. and Robinson, J. (1999) Live paper: video augmentation to simulate interactive paper. In Proceedings of the Seventh ACM international Conference on Multimedia (Part 2) (Orlando, Florida, United States, October 30 - November 05, 1999). MULTIMEDIA '99. ACM Press. [11] Sellen, A. and Harper R. (2001) The Myth of the Paperless Office. MIT Press, 2001. [12] Wellner, P. (1993) Interacting with paper on the DigitalDesk. Communication. ACM 36, 7 (Jul. 1993), 87-96. [13] Whittaker, S. and Hirschberg, J. (2001) The character, value, and management of personal paper archives. ACM Trans. Computer-Human. Interact. 8, 2 (Jun. 2001), 150-170.

1348