her actions. We will proceed as follows. First, we give a brief overview about several approaches currently used to support users in coping with high volume ...
Supporting Situated Actions in High Volume Conversational Data Situations Christopher Lueg
AI-Lab, Department of Computer Science University of Zurich Winterthurerstrasse 190, CH-8057 Zurich, Switzerland Tel. +41 1 63 54577 Fax +41 1 63 56809 lueg@i .unizh.ch
ABSTRACT
The global conferencing system Usenet news o ers an amount of articles per day that exceeds human cognitive capabilities by far although the articles are already organized in hierarchically structured discussion groups covering distinct topics. We report here on a situated information ltering system that signi cantly reduces the burden by supporting the user in acting situated. Interpreting the user's actions as situated actions, the approach complements current ltering and recommender approaches by completely avoiding the modeling of user interests; the user is the only instance for assigning (un-)interestingness to Usenet discussions.
Keywords
Situated cognition, situated actions, Usenet news, information ltering
INTRODUCTION
The huge and increasing amount of information available in the information age suggests to investigate new ways to support humans in gathering information that might be interesting, helpful, or necessary for them. Since the overall amount of information exceeds human cognitive capabilities by far, computers are increasingly used to help users to nd the information they are looking for. One of the central questions is how to provide adequate support for the information seeking process. From a cognitive science and situated cognition perspective, the goal is not to automate but to support this process in order to allow for situatedness and the peculiarities of human cognition. In this paper, we present a situated information ltering approach to support users in coping with high volume conversational data. This approach is based on
Proceedings of the Annual ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'98), April 18-23, 1998, Los Angeles, CA, USA.
the perspective that human behavior is inherently situated (see below) and complements other approaches by avoiding to automatically nd information that might be interesting to the user. Support is given by a potentially signi cant reduction of the amount of new Usenet articles that are to be investigated by the user. In addition, the user is supported in becoming aware of his or her interests. Data gained by monitoring the user's browsing behavior is used to nd out about discussions that are likely to be uninteresting (instead of trying to determine the interesting discussions, since this would involve modeling of the user's interests). In order to avoid misinterpretations, the situated information ltering approach uses a high degree of interactivity. The augmented newsreader interface always allows the user to accept or to reject indicated consequences of his or her actions. We will proceed as follows. First, we give a brief overview about several approaches currently used to support users in coping with high volume conversational data situations and other so-called \information overload" [18] situations. Then, we introduce the notion of situatedness and discuss some implications of this view for the design of human-computer interfaces. Based on these considerations, we discuss the bene ts of the situated information ltering approach and describe how the state-of-the-art newsreader Knews has been modi ed to support situated information ltering in the Usenet news domain. The paper concludes with a discussion of rst user experiences with the augmented newsreader which we call spynews because of the additional spying behavior.
CURRENT SUPPORT FOR INFORMATION SEEKING PROCESSES
Objects that are potentially interesting to users may vary from virtual entities, such as World Wide Web (WWW) pages or Usenet articles, to real-world entities, such as books, CDs, or movies. The former are accessible to computers while the latter typically are inaccessible to computers. Concerning the way objects are dealt with, basically two distinct approaches can
use of meta-data about objects (e.g., recommendations given by other users)
AAAAAAA AAAA AAAAAAA AAAA AAAAAAA AAAA AAAA AAAAAAA AAAA AAAAAAA AAAA AAAAAAA AAAAAA AAAAAAA AAAAAA AAAAAAA AAAAAAA AAAAAA AAAAAAA AAAAAAA AAAAAA AAAAAAA AAAAAAA AAAAAAA AAAAAAA AAAAAA AAAAAA AAAAAA Firefly (webpages, music, movies)
use of content analysis (e.g., information retrieval techniques)
PHOAKS (WWW)
Grouplens (USENET)
WestLaw (documents/ legal material)
INFOS (USENET)
Letizia (WWW)
INFOSCOPE (USENET)
Stanford Information Filtering Tool SIFT (USENET)
Standard killfiles (email, USENET)
Situated Information Filtering
use of data about the user (e.g., data gained by monitoring the user)
Figure 1: Classi cation of several systems according to the way they deal with the objects under investigation and the extent with which they rely upon monitoring user behavior (see text). currently be distinguished (see gure 1). On the one hand, traditional information ltering systems [23] directly deal with potentially interesting objects by analyzing these objects and extracting certain features indicating \interestingness". Commonly, these systems deal only with textual objects (i.e., documents), such as most WWW pages or Usenet articles, and incorporate information-retrieval techniques, which can be applied to information ltering [4]. Documents are dealt with by conversion into surrogates more amenable to automatic processing. Boolean, vector space, and probabilistic retrieval models are the main techniques in order to match models of user interests, commonly referred to as pro les, and models of documents [8]. The best matching documents are considered most promising and are presented to the user. Adaptive ltering systems use characteristic features of documents actually selected by the user to adapt the model of the user's interests to changes in these interests. Most current information-retrieval techniques are actually limited to textual representations although techniques for multimedia information retrieval are subject to research [19].
On the other hand, recommender systems [25], also referred to as social ltering systems or collaborative ltering systems, mimic the social process of giving recommendations to friends and colleagues in situations, when the information at hand is insucient to make a good choice. Recommender systems collect and aggregate recommendations given by other users in order to give advice upon request. One of the main advantages of this automation of the \word of mouth" [26] is that the recommendation approach is completely independent from the actual topic and the representation of the objects to be recommended. Therefore, recommender systems can deal with both virtual objects and real-world objects. Indeed, recommender systems exist for topics as di erent as movies, music, and documents, such as Usenet articles and most WWW pages. As in adaptive ltering systems, requests for advice and feedback on user satisfaction are used to construct a model of the user's interests. Figure 1 classi es several major approaches according to the way they deal with the objects under investigation and the extend with which the approaches rely upon monitoring user behavior. Examples for systems
that directly deal with textual objects without additional user data are standard \kill les" incorporating simple keyword-matching algorithms in order to hide or highlight certain emails or Usenet articles. Examples for systems incorporating this kind of ltering are the newsreaders Gnus1 and Knews2. A prime example for an advanced ltering system directly dealing with documents is the Stanford information ltering tool SIFT [34] doing brute-force ltering. Examples for systems that do not directly access the objects under consideration but deal with (meta-)data about these objects are the GroupLens3 [24, 12] collaborative ltering system for Usenet news, the PHOAKS4 collaborative ltering system [32, 33] for WWW resources, the Firefly5 recommender system, and the MORSE6 movie recommendation system. Letizia [15], INFOS [21], and INFOSCOPE [29] are examples for hybrid approaches dealing with both, data about the user's behavior and data extracted from the documents (WWW pages and Usenet articles, respectively). The GroupLens researchers are also investigating the incorporation of additional data about the user's reading behavior [20]. The situated information ltering approach shares a lot of similarities with these systems in that it incorporates data about the user's information seeking behavior. However, the situated information ltering approach [16] presented in this paper is di erent, since it completely avoids to analyze the potentially interesting objects and it does not incorporate modeling of the user's interests (see below).
ABSTRACTIONS AND SITUATEDNESS
Both, the recommender approach and the informationretrieval-based ltering approach, abstract away from the objects they are dealing with (e.g., pictures or documents). In the case of recommender systems, the abstraction is done when humans express their likes and dislikes in terms of numerical ratings and when these ratings are aggregated to preference pro les. Regarding the recommendations, the social context of a recommendation is abstracted away from its social embedding; the recommendation is de-contextualized [17]. In the case of information-retrieval-based ltering systems, the abstraction is done implicitly, since most information retrieval techniques deal only with representations and not with the pragmatic meaning the text has to the users. It is a prerequisite of these approaches that they construct an abstract representation of the user's interests, 1 http://www.gnus.org/ 2 http://www.student.nada.kth.se/ su95-kjo/knews.html 3 http://www.cs.umn.edu/Research/GroupLens/ 4 http://www.phoaks.com/ 5 http://www.firefly.com/ 6 http://www.labs.bt.com/innovate/multimed/morse/
morse.htm
commonly referred to as pro le. The pro le is necessary -within the chosen rationalistic approach [18]- to automatically match assumed user interests with documents potentially satisfying these interests. However, apart from its computational advantages, such as ecient matching of pro les with objects, the use of abstraction implies a gap between actual user interests and their abstract descriptions. From a situated cognition [6] perspective, it would be best to completely avoid this gap in order to allow for the peculiarities of human cognition. In contrast to the so-called \rationalistic" perspective, which views human cognition as data-processing and behavior as being largely predetermined by plans, the situated cognition perspective suggests to view cognition, knowledge, and behavior as being fundamentally situated: cognition and knowledge are emergent properties of the interaction of an individual with its environment, i.e., its current situation (thus, the term \situatedness"). Cognition cannot be reduced to internal \data-processing", it cannot be \de-contextualized" into a set of abstract descriptions [31, 6]. One important implication of situatedness is that the way a human interacts with a situation continuously changes based on his or her experience. Knowledge should not be viewed as substrate that can be extracted but as something that is located in physical interaction and social participation [6]. Accordingly, \interest" should be viewed as something that is dynamically generated, i.e., an emergent property of the interaction of an individual with an \information situation" [17]. This perspective, however, does not necessarily question the existence of so-called long-term interests which is a basic assumption in traditional information ltering [23]. The situated perspective demands new tools that support the user in acting situated in \information overload" situations instead of imposing restrictions or constraints on the user and his or her interests.
INFORMATION SEEKING AND SITUATED ACTIONS
Traditional adaptive information ltering is based on the assumption that certain human actions, such as selecting a particular document or spending a sucient amount of time reading a document, can be reasonably interpreted as indicators for \interest" in the corresponding document. However, even supposedly clear \expressions of interest" are always subject to the frame-of-reference-problem [5]. The frame-of-reference problem states that things might appear di erent from the user's and observer's perspective. In other words, the observer must be careful not to mix up observable behaviors with the user's internal mechanisms causing these behaviors. In case of information seeking behavior, the interest or information need is generated in the
head of the observer, rather than in the head of the observed subject. Also, the information need is not so much \inside the user's head" [4], but a result of the interaction of the user with a continuously changing situation. Results from research on the notion of interest indicate that it is hard to determine why a speci c document has actually been selected. Experiments revealed that explanations of why a document was chosen for reading, or why it was found to be interesting varied and changed over time. The same result was obtained when the subjects were asked about their initial information need [14, 21, 11]. Now, if the selection of a document is interpreted as a \situated action" some of the putative \inconsistencies" and \irregularities" in the observations reported in the literature can be explained. From the situated cognition perspective it can be explained why it is so hard to describe an information need. Information-need situations are dynamic and constantly changing [3]. Information needs cannot be reduced to internal information processes alone, but require interaction with the current information situation. This cognitive science perspective is in line with results from research on relevance also indicating that situational factors other than just the topical content of a selected document in uence relevance judgments. Factors in uencing the judgments are any factors that users bring into the situation, such as experience, background, knowledge level, beliefs, and personal preferences [3]. Also, the user judgments are in uenced by the purposes, expectations, the relevance of references, and future time savings [30].
SITUATED INFORMATION FILTERING
We have investigated new ways to support humans in high volume conversational data situations. Our situated information ltering approach focuses on supporting the user in acting situated rather than doing the information ltering for him or her. Avoiding to model the user's interests, our approach helps the user focus on what might be interesting by ltering what has shown to be uninteresting (see below). A prime example for high volume conversational data situations is participating in the global conferencing system Usenet news. Due to its high data volume and its global availability, Usenet is frequently used for information ltering experiments (e.g., [10, 27, 13, 21]). Usenet o ers more than 300000 articles per day7 and the amount is still increasing. Although the articles are already organized within a hierarchy of more than 15.000 newsgroups (groups of articles sharing a particular topic, e.g., the newsgroup comp.ai has the topic 7
Usenet trac statistics are published by UUNet in and other newsgroups.
de.admin.lists
computers and arti cial intelligence), it is not uncommon that high volume newsgroups still o er more than 1000 articles per day. Most Usenet users participate in several newsgroups covering di erent topics. It is not uncommon to scan 20-30 newsgroups for interesting articles. Typically, only a minor part of all articles in a newsgroup is actually read. From the observer's perspective, the main problem seems to be to detect the \interesting" articles among the uninteresting ones. Accordingly, most approaches to help users in coping with high volume conversational data or other \information overload" situations try to partly automate the information seeking process by matching models of user interests with potentially interesting documents (e.g., [10, 27, 13, 21, 15]). However, considering the frame-of-reference problem and the inherent dynamic of interests suggests to make as few assumptions as possible about the user's motivation and his or her actual interests. Consequently, the main di erence between the situated information ltering approach and more traditional information ltering approaches how the data about the user's behavior is interpreted and how the data is exploited for ltering. The situated information ltering approach does neither consider assumed interests of the users nor the content of selected documents. Instead, situated information ltering helps the user to focus on potentially interesting articles by reducing ( ltering) the amount of uninteresting data. The following design guidelines [18] helped to ensure that our approach appropriately accounts for situatedness: 1. Not too much value should be attributed to (single) user actions; they should not be interpreted as clear indicators of interest (sometimes, humans make faults, or they are under the in uence of events that do not directly relate to the document search; documents may turn out to be uninteresting after having selected them). 2. Selection of a document should not be interpreted as necessarily indicating an information need. 3. Not too much importance should be attributed to author, title, etc. 4. Information needs depend on the actual situation: they are dynamic, not static. For our Usenet experiments, the state-of-the-art Knews8 newsreader has been augmented in order to provide support for situated information ltering. The resulting spynews newsreader [16] monitors most user actions, such as selecting a newsgroup, selecting a discussion (commonly referred to as thread) within a news
8 http://www.student.nada.kth.se/ su95-kjo/knews.html
group, saving or printing an article, posting a followup, etc. Also, implicit data about the user's behavior is collected, such as the delay between the selection of a particular document and the selection of the next document (i.e., time supposedly spent to read the document) and whether the user has read a discussion partly or completely (i.e., whether all previously unread articles in the discussion have been read). Also, the user's basic browsing behavior is monitored. In order to inspect a particular discussion the user has to move a bar up or down to select the discussion among other discussions that are displayed in the main discussion window. Moving the bar can be accomplished by either using the arrow keys on the keyboard or pointing and clicking with the mouse. However, the data about the basic browsing behavior is not used for nding out about user interests but for gaining insights into the user's newsreading behavior, such as the detection of varying browsing habits in distinct newsgroups. First results indicate that the newsreading habits vary signi cantly between newsgroups and range from selecting only a few discussions to sequentially reading all available discussions. The situated information ltering approach is a discussion-oriented approach and di ers in this respect from most traditional ltering approaches focusing on single articles. Moreover, the situated information ltering approach focuses on the discussions the user is not interested in since evidence for not being interested in a discussion is much stronger than evidence for being interested in a discussion. The detection of uninteresting discussions can be accomplished by monitoring whether the user repeatedly ignores a discussion while browsing a newsgroup. While the user is moving the selection bar towards a discussion he or she is interested in, the bar will be be moved across other discussions. This does not necessarily mean that the user is not interested in these discussions at all but it is at least an indicator that the user is less interested in the discussion. Ignoring a discussion does not cause the discussion to vanish immediately but to fade out gradually according to the overall \attention" paid to this discussion. The \behavioral" status of a discussion is visualized as an augmentation to the representation of a discussion (in Knews a text line containing the subject of the discussion, the number of unread articles, and the author of the rst article). A little mark changes its shape according to the user's actions. Ignoring a discussion causes the mark (and the newsgroup, accordingly) to migrate towards its nal \uninteresting" state. Only if a discussion status reaches one of these states, the discussion will be marked as read or marked for ltering for a user-de ned period of time. These actions are executed automatically when the user leaves the
newsgroup. However, the user always has the opportunity to abandon the indicated consequences by clicking the third mouse button on a discussion which immediately resets the status of the discussion. Also, entering a discussion decreases the status since entering is interpreted as weak \interest". The complete advanced newsreader behavior can be controlled via userde nable parameters. These parameters do not only determine how often, for example, a subject has to be ignored in order to be ltered, but also allow an adaptation of the advanced newsreader behavior to di erent newsgroups since browsing varies signi cantly between di erent Usenet newsgroups. Put in a nutshell, the markers always indicate the consequences of the user's actions to ensure a comprehensible, predictable, and controllable user interfaces [28]. Leaving a newsgroup still containing unread discussions is interpreted as being less interested in the remaining discussions. Accordingly, the status of the discussions is slightly increased towards \uninteresting". However, if the user leaves the newsgroup by marking all unread discussions as read (the so-called catchup command) this is interpreted as stronger indicator for not being interested in the remaining discussions. Accordingly, the status of the discussions is more seriously increased compared to just leaving the newsgroup. The nal status of all discussions in the newsgroup is preserved in a database for further newsreading sessions. When the user (re-)enters the newsgroup, the status of the discussions is restored in order to reorder the discussions according to their supposed interestingness. Since no \interestingness" information about new discussions is available, these discussions are assumed potentially interesting and are located at the top. Below new discussions, the discussions that are already known are listed according to their status. Since no model of user interests is used to infer a degree of interestingness, the ordering of the discussions is decreasing, so that the less interesting discussions are located at the bottom of the window. This reordering of newsgroups between di erent sessions causes discussions behave like bubbles oating up and down according to the user's attention (in terms of actions) paid to the discussions. It is important to note that the status of a discussion is not determined on the basis of the content of the corresponding articles but the status is a result of the interaction of the user with the discussion. The situated information ltering approach exploits the dynamic nature of Usenet and its technical constraints [9]. Two di erent methods to relate incoming articles to discussions are available. On the one hand, articles can be grouped to discussions by comparing the Subject: entries since all articles with the same Subject: entry are assumed to belong to the same discussion. This pattern-matching approach has some se-
vere technical drawbacks, such as problems with entries that were not modi ed strictly according to the general Usenet rules in case of follow-ups or topical changes. On the other hand, considering a special References: entry allows to relate newly arriving articles to discussions (commonly referred to as threading). Knews and other state-of-the-art newsreaders already use references to organize articles to discussions. spynews as a discussion-oriented approach lters on the basis of both the subject and the references. spynews detects that the user ignores discussions on the basis of the subject but the actual ltering is done based on Knews's threading of the corresponding articles. The ltering of the subsequent articles is based on the assumption that follow-ups to uninteresting articles are also uninteresting [22]. However, technically \belonging to the same discussion" does not necessarily mean that follow-up articles still have the same topic as the original article since it is a peculiarity of Usenet discussions to keep a discussion title while the topic of the discussion has already shifted to another topic. According to the situated perspective, the decision whether this new topic under the old subject is still interesting should be completely left to the user. Therefore, ltering of discussions is limited in time. Also, it is left to the user whether split-o discussions (commonly referred to as subthreads) are to be ltered if the original discussion is already ltered.
DISCUSSION
In this paper, we have presented a situated information ltering approach to support users in coping with high volume conversational data situations in the Usenet domain. The approach accounts for situatedness by avoiding to model the user's interests or to interpret his or her actions as clear indicators of interest. Instead, help is provided to focus the user's attention on potentially interesting articles by gradually fading out, deleting, and reordering subjects that have shown to be less interesting. Depending on the actual newsgroup, this signi cantly reduces the amount of potentially interesting subjects the user has to browse. For our Usenet experiments, the state-of-the-art Knews9 newsreader has been augmented in order to provide support for situated information ltering. Experiences with the resulting spynews newsreader prototype have shown that it is indeed possible to provide an encouraging level of support without automating the information seeking process by matching models of user interests with documents. Experienced Usenet users nd this more situated approach useful. However, further experiments have to be conducted to clarify whether this also holds true for less experienced or even novel users.
9 http://www.student.nada.kth.se/ su95-kjo/knews.html
Users like the high degree of interactivity and the permanent visual feedback provided by the spynews interface since this feedback guarantees that the ltering is predictable and controllable. However, these results are preliminary since they only re ect the opinions of experienced Usenet users which were familiar with the standard Knews newsreader. Long-term user tests involving less experienced users and users preferring other newsreaders than Knews are under preparation and are scheduled for the next few months. The eciency of this discussion-oriented approach, however, depends on the average length (i.e., number of unread articles) of the uninteresting discussions. The length of Usenet discussions varies signi cantly between newsgroups. For example, \discussions" in announcement newsgroups are typically quite short, while discussions in newsgroups dealing with recreational topics or with topics related to computer security or Usenet administration may comprise hundreds of articles. It is one of the strengths of this discussion-oriented approach to release the user from long-lasting discussions that have shown to be uninteresting. Running spynews on low-end Sun workstations under SunOS and Solaris, experiences have shown that the additional database accesses at the beginning and at the end of browsing a newsgroup do not signi cantly in uence the runtime behavior of the spynews newsreader. Also, the size of such a database (currently one database per newsgroup the user is subscribed to) is still less than 100 kilobytes after several weeks of usage. The expire time (i.e., how long subjects are kept in the spynews database) has been set to one month. Although rst results are encouraging, they should not be interpreted as if this purely situated approach is an alternative to traditional ltering and recommendation approaches. Instead, the more situated approach should be regarded as a complement to traditional approaches. The golden mean is probably somewhere between a more rationalistic approach and this purely situated approach. Extended user experiments are under preparation to nd out about the extent to which the situated ltering perspective can be reasonably exploited. Currently, our focus is on Usenet only but we believe that the situated perspective is also valuable for other domains with \information overload" situations, such as email or the WWW. Also, the notion of situatedness introduces an important perspective into the technology-driven, largely rationalistic, Internet culture.
RELATED WORK
Although based on a di erent perspective, which is strongly in uenced by results from research on situated cognition, the situated ltering approach shares
similarities with traditional recommender or information ltering approaches. Similar to most advanced ltering and recommending approaches in the Usenet domain [7, 22, 21] or in the WWW domain [1, 15, 2], the situated information ltering system spynews exploits data gained \for free" [15] by monitoring the user's behavior. However, based on the user's actions, these approaches try to infer what the user is interested in. spynews is di erent in that it avoids to infer user interests. To the contrary, it supports the user in focusing on potentially interesting discussions by ltering discussions that have shown to be uninteresting. As in recommender systems, spynews does not directly deal with the objects under consideration. However, recommender systems abstract ratings to pro les while spynews avoids this abstraction step. The \interestingness" of discussions is solely based on the interaction of the user with the available discussions. Of course, user actions are interpreted in spynews, i.e., meaning is projected onto the actions, but the interpretations are not used to infer what the user is interested in. Also, the consequences of the interpretations are weak and interactivity ensures allows the user to accept or reject the indicated consequences. In particular, spynews shares basic assumptions with two of the above mentioned systems, INFOSCOPE [7] and Letizia [15]. Both systems aim at supporting the user instead of completely automating the task at hand. Both systems try to make the best use of the most limited resource of the user, which is the user's attention. Also, both systems try to keep the context of their recommendations: Letizia provides on-line support while the user is browsing the WWW and INFOSCOPE regroups Usenet articles to arti cial special-interest newsgroups. However, spynews completely avoids the modeling of interests.
ACKNOWLEDGMENTS
[2] Marko Balabanovic. An adaptive web page recommendation service. In Proceedings of the First International Conference on Autonomous Agents, 1997. [3] Carol L. Barry. User-de ned relevance criteria: An exploratory study. Journal of the American Society for Information Science (JASIS), 45(3):149{ 159, 1994. [4] Nicholas J. Belkin and W. Bruce Croft. Information ltering and information retrieval: Two sides of the same coin? Communications of the ACM, 35(12):29{38, December 1992. [5] William J. Clancey. The frame of reference problem in the design of intelligent machines. In K. van Lehn, editor, Architectures for Intelligence. The 22nd Carnegie Mellon Symposium on Cognition, pages 257{423. 1991. [6] William J. Clancey. Situated Cognition. On Human Knowledge and Computer Representations. Cambridge University Press, 1997. [7] Gerhard Fischer and Curt Stevens. Information access in complex, poorly structured information spaces. In Proceedings of the Annual ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'91), pages 63{70. ACM
[8]
[9] [10]
The author would like to thank Ralf Salomon and Elke Siemon for their help and is grateful to Rolf Pfeifer for a stimulating research environment. Special thanks go to Karl-Johan Johnsson for the permission to use and to modify his Knews newsreader for the Usenet experiments.
[11]
References
[12]
[1] Robert Armstrong, Dayne Freitag, Thorsten Joachims, and Tom Mitchell. Webwatcher: A learning apprentice for the world wide web. In
Information gathering from heterogeneous, distributed environments : Papers from the 1995 AAAI Spring Symposium, Menlo Park, California,
1995. AAAI Press.
[13]
Press, April 1991. Peter W. Foltz and Susan T. Dumais. Personalized information delivery: An analysis of information ltering methods. Communications of the ACM, 35(12):51{60, December 1992. M. Horton and R. Adams. RFC-1036 (request for comments), December 1987. Andrew Jennings and Hideyuki Higuchi. A personal news service based on a user model neural network. IE-ICE Transactions on Information Systems, March 1992. Fredrik Kilander, Eva Fahrus, and Jacob Palme. Intelligent information ltering. Technical report, Department of Computer and Systems Science, Stockholm University, February 1997. Joseph A. Konstan, Bradley N. Miller, David Maltz, Jonathan L. Herlocker, Lee R. Gordon, and John Riedl. Applying collaborative ltering to Usenet news. Communications of the ACM, 40(3):77{87, March 1997. Ken Lang. Newsweeder: Learning to lter netnews. In Proceedings of the Twelth International Conference on Machine Learning. Morgan Kaufmann, 1995.
[14] Ann Lantz. How do experienced users of the system Usenet news select their information? Technical report, Department of Computer and Systems Science, University of Stockholm, 1993. [15] Henry Lieberman. Autonomous interface agents. In Proceedings of the Annual ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'97), pages 67{74. ACM Press, March 1997.
[16] Christopher Lueg. An adaptive Usenet interface supporting situated actions. In Proceedings of the 3rd ERCIM Workshop on User Interfaces for All, 1997. [17] Christopher Lueg. Social ltering and social reality. In Proceedings of the 5th DELOS Workshop on Filtering and Collaborative Filtering, 1997. [18] Christopher Lueg and Rolf Pfeifer. Cognition, situatedness, and situated design. In Jonathan P. Marsh, Chrystopher L. Nehaniv, and Barbara Gorayska, editors, Proceedings of the Second International Conference on Cognitive Technology, pages 124{135. IEEE Computer Society, 1997. [19] Mark T. Maybury, editor. Intelligent Multimedia Information Retrieval. MIT Press, 1997. [20] Bradley N. Miller, John T. Riedl, and Joseph A. Konstan. Experiments with GroupLens: Making Usenet useful again. In Proceedings of the 1997 USENIX Winter Technical Conference, January 1997. [21] Kenrick J. Mock. Hybrid hill-climbing and knowledge-based techniques for intelligent news ltering. In Proceedings of the National Conference on Arti cial Intelligence (AAAI'96), Menlo Park, California, 1996. AAAI Press. [22] Masahiro Morita and Yoichi Shinoda. Information ltering based on user behavior analysis and best match text retrieval. In Proceedings of the ACMSIGIR Conference on Research and Development in Information Retrieval (SIGIR'94). Springer-
Verlag London, 1994. [23] Douglas W. Oard and Gary Marchionini. A conceptual framework for text ltering. Technical Report CS-TR-3643, University of Maryland, May 1996. [24] Paul Resnick, Neophytos Iacovou, Mitesh Suchak, Peter Berstrom, and John Riedl. GroupLens: An open architecture for collaborative ltering of netnews. In Richard Furuta and Christine Neuwirth, editors, Proceedings of the International Conference on Computer Supported Cooperative Work
(CSCW'94), pages 175{186. ACM Press, October
1994. [25] Paul Resnick and Hal R. Varian. Recommender systems. Communications of the ACM, 40(3):56{ 58, March 1997. [26] Upendra Shardanand and Pattie Maes. Social information ltering: Algorithms for automating \word of the mouth". In Irvin Katz et al., editor, Proceedings of the Annual ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'95), pages 210{217. ACM Press, May 1995.
[27] Beerud Sheth and Pattie Maes. Evolving agents for personalized information ltering. In Ninth Inter-
national Conference on Arti cial Intelligence for Applications, pages 345{352. IEEE Computer So-
[28]
[29] [30]
[31] [32]
[33]
ciety Press, March 1993. Ben Shneiderman. Direct manipulation for comprehensible, predictable, and controllable user interfaces. In Proceedings of the International Conference on Intelligent User Interfaces (IUI'97), pages 33{39, January 1997. Curt Stevens. Automating the creation of information lters. Communications of the ACM, 35(12), December 1992. Louise T. Su. The relevance of recall and precision in user evaluation. Journal of the American Society for Information Science (JASIS), 45(3):207{ 217, 1994. Lucy Suchman. Plans and situated actions The Problem of Human-Machine Communication. Cambridge University Press, 1987. Loren Terveen, Will Hill, Brian Amento, David McDonald, and Josh Creter. PHOAKS: A system for sharing recommendations. Communications of the ACM, 40(3):59{62, March 1997. Loren Terveen, William C. Hill, Brian Amento, David McDonald, and Josh Creter. Building taskspeci c interfaces to high volume conversational data. In Proceedings of the Annual ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'97) Conference Proceedings, pages 226{
233. ACM Press, 1997. [34] Tak W. Yan and Hector Garcia-Molina. SIFT - a tool for wide-area information dissimination. In 1995 USENIX technical conference, pages 177{ 186, 1995.
