Ann Lantz Dpt of Computer and System Science Stockholm University Electrum 230, S-164 40 Kista, Sweden +46 8 16 16 ?? alz@dsv.su.se

ABSTRACT

Text of abstract. KEYWORDS: Text classification, information filtering, computer-mediated communication.

INTRODUCTION

Keywords and phrases are in common use in information filtering ([4, 5, 9]), but other criteria could be usable, especially in combination with machine learning techniques. By studying how users of Usenet News select and rate information, we have searched for additional filtering criteria.

USER STUDIES

In a first pilot study [2] the aim was to investigate how experienced users of Usenet News selected their information and how they reacted towards assisting the system in different ways i. e. by rating contributions, write key words, use rules or by letting the system take part of the users profile. Four subjects were instructed to perform a set of tasks in Usenet News and think aloud during the session. After the session the subjects were interviewed.

The results pointed at the fact that the users selected their information by taking notice of the information in the message header; mainly key words they saw in the title, and partly by browsing through the text, where keywords and general layout are used for selection. In an unfamiliar news group the selection took longer time, apparently since the keywords were unfamiliar. The subjects were positive to rating contributions, adding keywords and also to use rules. On the question if the system should be allowed to take part of the users profile, one subject expressed the feeling of need for privacy i. e. not wanting the system to know all of the groups he was entering.

In a second pilot study [3] the aim was to investigate the criteria for selecting information used by experienced readers of Usenet News. Three subjects were instructed to select ten contributions from five newsgroups each and to think aloud while they read. They were asked to read the contributions and place each in one of three categories (read now, save, discard).

The subjects used depth of comment, keywords, patterns in the contribution, amount of cited text, questions, length of text, alphabetical signs or numbers to form their decicions. These criteria have been devided into two groups; pattern recognitions and demands reading.

A questionnaire have recently been distributed in a Swedish Usenet News newsgroup. The aim with the study was to answer the following questions:

• do users of Usenet News experience a need for filtering?

• which criterias do the users adopt to find or avoid information in Usenet News?

• in what way are the users willing to support the system?

• how important do the users regard the information presented in Usenet News?

• do the users experience that they have enough time to spend with the news system?

Twenty subjects were included in the study and the preliminary results show that:

• the need for filtering were experienced by 19 of the subjects and they did not express a need for filtering information in the meetings not subscribed.

• criteria

TECHNICAL WORK

In order to facilitate automatic recognition of messages that the user is likely to regard as interesting, some kind of feedback model is required. The feedback model enables the user to provide the system with information about preferred messages. One of the most uncomfortable ways to do this, is to force the user to formulate such preferences in terms of a rule system with rigid and narrow semantics. Popular readers for Usenet News commonly employs such rules. With them, the burden of formulation into operative and non-conflicting rules is put on the human.

Other schemes has been proposed where the user supplies either an explicit or implicit rank on each message. The rankings are then treated as classification labels and machine learning techniques are employed to distill an operative and efficient distinction. Explicit ranking of messages, however, distracts the user from the primary task of reading. The burden is attenuated by inductive machine learning techniques which commonly require many correctly and consistently ranked examples and counterexamples. Implicit ranking does not put any extra strain on the user, but must make a number of ad hoc assumptions with respect to the behaviour of each user. Such

Message Filtering for Preference and Serendipity

assumptions include the expected reading time per lines of text, that cited text is not read in detail, that the user is actually reading and not answering the telephone and that the user does not reject relevant messages by accident.

We are of the opinion that the user is willing and able to supply an automatic classification system with a small number (five or less) of prototypical messages for each conceptually distinct and interesting topic. We also believe that the user is prepared to (although not required) review the installed prototypes in order to delete text, add important fragments or emphasize critical parts of the existing text.

Our approach to the problem is to view it as an information retrieval process, where the user has a number of outstanding queries. Each query corresponds to a topic the user is interested in. The document collection at any time, is the sequence of unread messages provided by the subscribed newsgroups. This view has earlier been presented by Belkin and Croft [1].

Our work is similar to that of Sheth [8], Morita and Shinoda [6], and Yan and Garcia-Molina [10]. Like them, we have decided to use a simple statistical similarity measure for queries and documents. We have evaluated two such measures, the well-known cosine measure (Salton [7]), and sub-string indexing (Morita and Shinoda [6]).

However, in response to empirical studies by Lantz on readers of Usenet News, we also want to take advantage of the structural information contained in each message. We believe that a message filter (in contrast to an information filter) should be able to gain discriminative power from the physical appearence of a message. In quantitative terms: the amount of cited text, the layout of the text, the proportion of misspelled words or

References

[1] Nicholas J. Belkin and Bruce W. Croft. Information filtering and information retrieval: Two sides of the same coin? Communications of the ACM, 35(12):29– 38, December 1992.

[2] A. Lantz. How do experienced users of the system usenet news select their information? Working paper in the IntFilter project, 1993.

[3] A. Lantz. Useful criterias for intelligent filtering? Working paper in the IntFilter project, 1994.

[4] T. W. Malone, K. R. Grant, F.A. Turbak, S.A. Brobst, and Cohen D. Intelligent information.sharing systems. Communications of the ACM, 30(5):390–402, 1987.

[5] T.W. Malone, K-Y. Lai, and C. Fry. Experiments with oval: A radically tailorable tool for cooperative work. In CSCW Proceedings, pages 289–297, 1992.

[6] Masahiro Morita and Yoichi Shinoda. Information filtering based on user behavior analysis and best match text retrieval. In Proceedings of the Seventeenth Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval, pages 272–281. Springer-Verlag, 1994.

[7] G. Salton and M. J. McGill. Introduction to Modern Information Retrieval. McGraw-Hill, 1983.

[8] B. Sheth. A learning approach to personalized information filtering. Master’s thesis, Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 1994.

[9] B. Sheth and P. Maes. Evolving agents for personalized information filtering. In Proceedings of the ninth IEEE Conference on Artificial Intelligence for applications, 1993.

[10] T. W. Yan and H. Garcia-Molina. Index structures for information filtering under the vector space model. Technical report, Dpt. of Computer Science, Stanford University, Stanford CA 94305, 1993.