Introduction to IR

Mike Salampasis

​

​

(to some extent based on slides from Dr Allan Hanbury and Prof. Bruce Croft)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1

Contents

• Information Retrieval
• How does a search engine work?
• How people search
• Interfaces for IR

​

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2

IBM Watson and Jeopardy

http://www.youtube.com/watch?v=oZZ9JxlkM3I

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3

The end of the show

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4

How does it work?

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5

How does it work?

Natural Language Processing

Information Retrieval

Knowledge Representation and Reasoning

Machine Learning

Parallel and Distributed Computing

Hypothesis�Generation

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

6

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7

Information Retrieval

Information Retrieval (IR) is finding material (usually documents) of an unstructured nature (usually text) that satisfies an information need from within large collections (usually stored on computers).

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

8

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

10

Why Information Retrieval?

• Information Overload
• “The Indexed Web contains at least 60 billion pages

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

11

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

12

Manual Lists of Websites

December 1996

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

13

Automatic Indexing

​

​

​

​

1996

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

14

Automatic Indexing

November 1998

June 2009

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

15

Beyond the Web

• Information is exploding in specialised domains too

Number of images captured per day in a large hospital (each slice of a volume is counted as an image), �~100 GB new images per day

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

16

The picture can be even more fascinating

• When you search, Google tries to figure out not just what you’re typing into the box, but what you mean. So algorithms for spelling, autocompletion, synonyms, and query understanding jump into action.
• When Google thinks it knows what you want, it pulls results from those billion of web pages pages, but it doesn’t just give you what it finds.
• First, a ranking procedure uses over 200 closely guarded secret factors that look at the freshness of the results, quality of the website, age of the domain, safety and appropriateness of the content, and user context like location, prior searches, Google+ history and connections, and much more.
• Then, in just over an eighth of a second, Google then delivers the results to your computer, tablet, or phone.

​

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

17

Search Influences Society

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

18

Basic assumptions of �Information Retrieval

• Collection: Fixed set of documents
• Goal: Retrieve documents with information that is relevant to the user’s information need and helps the user complete a task
• Information needs do not change as a consequence of observing the search results
• Relevance of a document is independent of the relevance of another document

​

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

19

Four Key Characteristics of IR

• Unstructured information
• No right answers
• Separation of indexing and query time processing
• Strong empirical method

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

20

Unstructured Information

• Text
• Images
• Music
• Videos

As opposed to

• Relational databases
• Lists of numbers

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

21

IR vs. Databases: Structured vs Unstructured Data

• Structured data tends to refer to information in “tables”

Employee

Manager

Salary

Smith

Jones

50000

Chang

Smith

60000

50000

Ivy

Smith

Typically allows numerical range and exact match

(for text) queries, e.g.,

Salary < 60000 AND Manager = Smith.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

22

Unstructured Data

• Typically refers to free text
• Allows
• Keyword queries including operators
• More sophisticated “concept” queries e.g.,
• find all web pages dealing with drug abuse
• Classic model for searching text documents

​

​

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

23

Semi-structured Data

• In fact almost no data is “unstructured”
• For example:
• This slide has distinctly identified zones such as the Title and Bullets
• Patents contain Title, Abstract, Description and Claims sections
• Facilitates “semi-structured” search such as
• Title contains data AND Bullets contain search

​

​

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

24

Confluence of IR and Databases

• Combine structured and unstructured search
• Find a scientific paper published after 2010 discussing diversification and search
• Use information extraction from text to populate databases
• Extract companies buying other companies from news articles and store them in a structure form
• Indexing of databases – Search-based applications
• Dump a database as xml and index this using IR algorithms

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

25

Unstructured Text Information in Practice

• e-mails
• memos
• notes from call centers and support operations
• news
• user groups
• chats
• reports
• tweets
• web pages
• blogs
• patents
• scientific papers
• …

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

26

No Right Answers

• Queries typically are lists of words or Boolean expressions
• Lack clear semantics: compare SQL
• Relevance - list of results judged to be relevant
• Exhaustive list of relevant results undeterminable or a matter of opinion
• In general, known-item search is an exception

​

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

27

Separation of Indexing and Query Time Processing

• IR is about large scale data collections
• The collection of information cannot be searched directly in interactive time
• Therefore we need to separate the process into:
1. Offline (crawl/index) time processing
2. Online query time processing

​

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

28

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

29

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

30

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

31

Empirical Method

• Need to prove whether one system is better than another
• Better systems produce more relevant information
• We need reproducibility
• Evaluation is required

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

32

Search Interface

• Almost all IR systems are accessed through a search box
• There is usually also an advanced search option

​

​

​

​

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

33

Results

• Results are almost always viewed as a vertical list

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

34

Why are interfaces so simple?

• Search is a means towards some other end, rather than a goal in itself
• Search is a mentally intensive task
• Nearly everyone who uses the web uses search

​

• Therefore the interface should be �non-distracting, non-intrusive and understandable

M. Hearst

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

35

How does a search engine �work?

• You have a collection of documents that you want to be able to search through
• What do you do?
• What steps are involved?

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

36

Conceptual Model for Search

Documents

Document Representation

Information Need

Query

Indexing

Formulation

Retrieved Documents

Retrieval Function

Further Analysis of the Documents

Relevance Feedback, Query Reformulation, Query Expansion

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

37

Conceptual Model for Search

Documents

Document Representation

Information Need

Query

Indexing

Formulation

Retrieved Documents

Retrieval Function

Further Analysis of the Documents

Relevance Feedback, Query Reformulation, Query Expansion

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

38

Types of Queries

• Boolean
• Brutus AND Caesar
• disabl! /p access! /s work-site work-place (employment /3 place)
• Free text queries
• Brutus Caesar
• requirements disabled people access workplace

​

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

39

Queries

• But what goes on once you click search can be very different...
• It depends on whether the system implements a Boolean search or a ranked search model
• The way that the query is formulated depends on the way the system works

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

40

Conceptual Model for Search

41

Documents

Document Representation

Information Need

Query

Indexing

Formulation

Retrieved Documents

Retrieval Function

Further Analysis of the Documents

Relevance Feedback, Query Reformulation, Query Expansion

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

41

Indexing

• How an IR system DOES NOT work:
• The user types in a query
• Then the system scans through all documents and returns those that match the query
• This would not allow rapid searching
• For this reason, the system first runs an indexing stage before any querying can be done

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

42

Aim of Indexing

• Storage of information in a way that supports efficient retrieval
• Two main points of consideration:
• Accuracy of representation
• Space and time efficiency
• The basic indexing process is pretty much the same for all search engines

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

43

Overview of Indexing Process

• Basic Concept

I like to laugh. It is a tonic. It braces me up—makes me feel fine!—and keeps me in prime mental condition. Laughter is a physiological necessity. The nerve system requires it. The deep, forceful chest movement in itself sets the blood to racing thereby livening up the circulation—which is good for us.

Without a word, Mr. Stevens caught up the tray from the piano and glided away on his toe-points; whereupon Mr. Brimberly (being alone) became astonishingly agile and nimble all at once, diving down to straighten a rug here and there, rearranging chairs and tables; he even opened the window and hurled two half-smoked cigars far out into the night;

It was always night on Martha, but Mark broke up his time into mornings, afternoons and evenings. Their life followed a simple routine. Breakfast, from vegetables and Mark's canned store. Then the robot would work in the fields, and the plants grew used to his touch.

The whole edifice bears the same warm tinge of yellow that all those of good quality acquire from age in that pure climate.

The untiring efforts of genius for over a century have succeeded in producing a musical instrument that falls little short of perfection.

Document Collection

Index

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

44

Overview of Indexing Process

I like to laugh. It is a tonic. It braces me up—makes me feel fine!—and keeps me in prime mental condition. Laughter is a physiological necessity. The nerve system requires it. The deep, forceful chest movement in itself sets the blood to racing thereby livening up the circulation—which is good for us.

Without a word, Mr. Stevens caught up the tray from the piano and glided away on his toe-points; whereupon Mr. Brimberly (being alone) became astonishingly agile and nimble all at once, diving down to straighten a rug here and there, rearranging chairs and tables; he even opened the window and hurled two half-smoked cigars far out into the night;

It was always night on Martha, but Mark broke up his time into mornings, afternoons and evenings. Their life followed a simple routine. Breakfast, from vegetables and Mark's canned store. Then the robot would work in the fields, and the plants grew used to his touch.

The whole edifice bears the same warm tinge of yellow that all those of good quality acquire from age in that pure climate.

The untiring efforts of genius for over a century have succeeded in producing a musical instrument that falls little short of perfection.

Document Collection

Index

Retrieval Model

User Interface

Retrieval System

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

45

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

46

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

47

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

48

Νόμος Zipf

ΑΤΕΙΘ - Τμήμα Πληροφορικής

49

25/02/2022

Ιεράρχηση x Συχνότητα ≈ Σταθερά

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

49

A plot of the rank versus frequency for the first 10 million words in 30 Wikipedias (dumps from October 2015) in a log-log scale.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

50

A Glimpse at �Document Representations

• Represent documents via the complete set of terms

I like to laugh. It is a tonic. It braces me up—makes me feel fine!—and keeps me in prime mental condition. Laughter is a physiological necessity. The nerve system requires it. The deep, forceful chest movement in itself sets the blood to racing thereby livening up the circulation—which is good for us.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

51

Index Creation

I like to laugh. It is a tonic. It braces me up—makes me feel fine!—and keeps me in prime mental condition. Laughter is a physiological necessity. The nerve system requires it. The deep, forceful chest movement in itself sets the blood to racing thereby livening up the circulation—which is good for us.

Without a word, Mr. Stevens caught up the tray from the piano and glided away on his toe-points; whereupon Mr. Brimberly (being alone) became astonishingly agile and nimble all at once, diving down to straighten a rug here and there, rearranging chairs and tables; he even opened the window and hurled two half-smoked cigars far out into the night;

It was always night on Martha, but Mark broke up his time into mornings, afternoons and evenings. Their life followed a simple routine. Breakfast, from vegetables and Mark's canned store. Then the robot would work in the fields, and the plants grew used to his touch.

The whole edifice bears the same warm tinge of yellow that all those of good quality acquire from age in that pure climate.

The untiring efforts of genius for over a century have succeeded in producing a musical instrument that falls little short of perfection.

Document Collection

Index

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

52

Direct Index

• Scales badly with respect to number of documents

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

53

Inverted Index

• Default index structure in Information Retrieval
• Computationally very efficient. Scales well
• Words are sorted alphabetically to speed up access
• Frequency of a word in a document can also be stored

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

54

Positional Index

• Stores the position of term in addition to the frequency
• What does this allow?
• Allows for word-order, phrasal, and offset querying

​

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

55

Field Index

• Enables part specific searching
• Allows assigning weight to different parts or aspects of documents
• A form of partitioning documents

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

56

Inverted Index Construction

Documents to

be indexed

Friends, Romans, Countrymen.

Sec. 1.2

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

57

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

58

Tokenization

• Input: “Friends, Romans, Countrymen”
• Output: Tokens
• Friends
• Romans
• Countrymen
• A token is an instance of a sequence of characters
• Each such token is now a candidate for an index entry, after further processing

Sec. 2.2.1

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

59

Tokenization

• Issues in tokenization:
• Finland’s capital →

Finland? Finlands? Finland’s?

• Hewlett-Packard → Hewlett and Packard as two tokens?
• state-of-the-art: break up hyphenated sequence.
• co-education
• lowercase, lower-case, lower case ?
• San Francisco: one token or two?
• How do you decide it is one token?

Sec. 2.2.1

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

60

Numbers

• 3/20/91 Mar. 12, 1991 20/3/91
• 55 B.C.
• B-52
• My PGP key is 324a3df234cb23e
• (800) 234-2333
• Often have embedded spaces
• Older IR systems may not index numbers
• Will often index “meta-data” separately
• Creation date, format, etc.
• Formulae (mathematical, chemical) are very difficult to index

Sec. 2.2.1

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

61

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

62

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

63

Normalization to terms

• We need to “normalize” words in indexed text as well as query words into the same form
• We want to match U.S.A. and USA
• Result is terms: a term is a (normalized) word type, which is an entry in the IR system dictionary
• We most commonly implicitly define equivalence classes of terms by, e.g.,
• deleting periods to form a term
• U.S.A., USA → USA
• deleting hyphens to form a term
• anti-discriminatory, antidiscriminatory → antidiscriminatory

Sec. 2.2.3

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

64

Lemmatization

• Reduce inflectional/variant forms to base form
• For example:
• am, are, is → be
• car, cars, car's, cars' → car
• the boy's cars are different colors → the boy car be different color
• Lemmatization implies doing “proper” reduction to dictionary headword form

Sec. 2.2.4

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

65

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

66

Stemming

• Reduce terms to their “roots” before indexing
• “Stemming” suggests crude affix chopping
• language dependent
• e.g., automate(s), automatic, automation all reduced to automat.

for example compressed

and compression are both

accepted as equivalent to

compress.

for exampl compress and

compress ar both accept

as equival to compress

Sec. 2.2.4

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

67

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

68

Conceptual Model for Search

Documents

Document Representation

Information Need

Query

Indexing

Formulation

Retrieved Documents

Retrieval Function

Further Analysis of the Documents

Relevance Feedback, Query Reformulation, Query Expansion

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

69

Information Retrieval Models

• A retrieval model is a mathematical, potentially probabilistic, model to rank retrieved documents
• Tasks of IR models:
• Process a query such that the result is specific (not too many hits and hits on topic) while being exhaustive (enough hits, good coverage)
• Retrieve relevant documents while not retrieving non-relevant documents
• Rank documents

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

70

Two Main Classes of IR Model

• Boolean Retrieval Model
• Extended Boolean Retrieval Model
• Ranked Retrieval Model
• Vector space model (VSM)
• BM25 / Okapi
• Language Modelling

​

​

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

71

How People Search

• Search can be done in many different ways
• User interaction with search interfaces differs depending on
• the type of task
• the domain expertise of the information seeker
• the amount of time and effort available to invest in the process
• Types of search task
• Direct Search – Known-Item Search
• Information Lookup
• Exploratory Search
• Browsing
• Exhaustive Searching

​

​

​

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

72

Direct search�Known-Item search

The user is searching for an information object which is already known to him or her.

​

• Well defined information need
• Exact extraction – know where to look
• More or less a non-iterative process
• High precision

​

• Well supported by web search engines

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

73

Information Lookup

The user is searching for facts or answers to questions.

​

• Well defined information need
• Satisfied by short, discrete pieces of information: numbers, dates, names or names of files or web sites
• Usually not iterative – the user scans through the list of returned results to find the answer or a link to a page containing the answer

​

• Generally well supported by web search engines

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

74

Exploratory Search

The user is seeking to learn something about a topic but does not know in advance what may be important.

• The information need is more unclear
• Snowball searching
• start with more known items and use references, citations to find more information objects
• very iterative process
• Precision focus
• Require a small information space so the results can be easily reviewed
• Automatic support could be query expansion, aggregated processing of the result set

​

​

​

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

75

Browsing

The goal is unclear since the user is not sure whether the requirements can be met or how they might be met.

​

• General
• to refine the users perception of their information need
• Purposive
• user has some fairly good idea what he is looking for

​

​

• Supported by the linking mechanism

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

76

Exhaustive Searching

The user is trying to learn everything about a particular topic

​

• Require high recall as well as precision
• Typical search within law (patent), medicine and intelligence
• Information sources analysis
• Consulting guides, classification schema, thesaurus
• Conducting domain analysis
• Background knowledge is essential for information satisfaction

​

• Limited support by existing IR systems

​

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

77

Interfaces for IR

• The search user interface should aid users
• In the expression of their information needs
• In the formulation of their queries
• In selecting among available information sources
• In the understanding of their search results
• In keeping track of the progress of their information seeking efforts

​

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

78

Interfaces for IR

• Almost all IR systems are accessed through a search box
• There is usually also an advanced search option

​

​

​

​

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

79

There is usually also an advanced search option

​

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

80

Results

• Results are almost always viewed as a vertical list

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

81

Visualization

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

82

Visualization

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

83

User Interface Characteristics

• Some characteristics of user interfaces that are found to be helpful are shown:
• Document surrogates
• Related term suggestion
• Faceted metadata

​

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

84

Document Surrogates

• A document surrogate contains information on why the document was retrieved, such as:
• Title
• URL
• Textual summary (also called snippets, extracts or abstracts)
• Search terms should be highlighted
• The design of document surrogates is an active research area
• The quality of the surrogate can affect the perceived relevance
• Today, summaries are designed to show the query terms in the context in which they occur in the document
• Tradeoff between long surrogates and space on the page
• Show parts where the query words occur close together
• Showing full sentences or parts of sentences?

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

85

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

86

http://biosearch.berkeley.edu

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

87

Related Term Suggestion

• Related term suggestion, or term expansion usually means the suggestion of alternative wording
• Some query term suggestions are based on the entire search session of the particular user
• Others are based on behavior of other users who have issued the same or similar queries in the past
• One strategy is to show similar queries by other users
• Another is to extract terms from documents that have been clicked on in the past by searchers who issued the same query

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

88

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

89

Suggestion Provenance

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

90

Faceted Metadata

• Faceted metadata consists of a set of categories (flat or hierarchical), each of which corresponds to a different facet (dimension or feature type) of the collection of items
• Faceted metadata allow the assignment of multiple categories to a single item
• An interface using faceted metadata is known as faceted navigation
• Query previews show how many documents will be returned when a user clicks on a facet

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

91

http://orange.sims.berkeley.edu/cgi-bin/flamenco.cgi/nobel/Flamenco

Flamenco search engine searching on Nobel Prize Winners

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

92

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

93

http://www.yelp.com

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

94

Comprehensive search interfaces

• Different user interfaces could be necessary for different types of search
• For example, in search for scientific papers on which to base a report
• The search could last a number of hours or days
• An interface with query history and the possibility to store results could be useful

​

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

95

Example: ezDL

• ezDL – meta search engine for multiple collections - http://www.ezdl.de

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

96

Summary

• Information Retrieval algorithms can be used to counter information overload
• They can form the basis of Question Answering applications
• End users can search in many ways
• IR interfaces can take many forms, some in which queries are not even required

​

​

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

97