Introduction to Information Retrieval

Background reading: Croft-Metzler-Stroman-Search_Engines_Information_Retrieval_in_Practice.md

Salton is considered to be a founding father of IR, and proposed the following early definition:

Information retrieval is a field concerned with the structure, analysis, organization, storage, searching, and retrieval of information - Gerard Salton, 1968.

A more recent definition states that IR concerns retrieving the right information, at the right time, in the right form.

Information objects are not necessarily textual, and web search is not the only application of IR.

Dimensions of IR

Information media:

  • Text (primary focus of the field)
  • Images (e.g. legal, medical, scanned documents)
  • Audio (music and speech)
  • Video

Applications

  • Vertical search (e.g. product search, travel search)
  • Enterprise search
  • Desktop search
  • Chatbots and conversational agents

Tasks

  • Ad hoc search
  • Filtering and classification
  • Question answering and passage retrieval

IR Core Issues

Relevance

Loose definition:

"A relevant document contains the information that a person was looking for when she submitted a query to the search engine."

What are relevance criteria?

  • Topicality: is the document about the requested information?
  • Readability: Is the text understandable?
  • Authoritiveness: Is the text trustworthy?
  • Child-suitability
  • Etc.

The above definition was at least partly subjective. Do we define relevance topically or merely with respect to the user?

What is the distinction between topical relevance and user relevance?

Insight: Users are a central element in IR; this field is not simply about the engineering part of e.g. "search".

Retrieval models address the issue of relevance in different ways.

Evaluation

How do we compare e.g. ranking algorithms?

Precision and recall are early measures

It is common to do offline testing on test collections, where the "relevant" documents are recorded for a set of queries. Experts are uses for recording relevant judgments.

  • Text REtrieval Conference (TREC) provides high-quality test collections
    • http://trec.nist.gov/

Online evaluation: e.g. how do users click on certain results?

Information need

IR involves users and their information needs.

Queries are very poor representations of users' information needs.

Getting the actual information need of the user is very difficult.

E.g. if we query "Cambridge population", how do we know if it's about Cambridge USA or Cambridge UK? Ambiguity is just one example of why query and information need may diverge.

IR explores several techniques to close the gap between query and information need:

  • Query expansion
  • Clarification questions
  • Query suggestions

Q: Are queries good representations of user needs? If yes, why? If not, give an argument why queries and user needs would easily diverge.

Q: What are some of the ways in which we can close the gap between a query and a user's information need?

Search Engines

Search engines are large scale IR applications.

Two distinct main goals:

  • Effectiveness (quality): concerns retrieval of the most relevant results; high quality
    • Often confused with efficiency
  • Efficiency (throughput): retrieve those results as quickly as possible.

Q: What is the difference between effectiveness and efficiency?

Architecture

Indexing process

Query process

  • Indexing procedure to build the structure enabling search (offline)

    • Text acquisition: make available documents that will be searched (crawler)
    • Document data store: store text and metadata of documents
    • Text transformation: transform documents into index terms of features
    • Index creation: for fast searching and retrieval
  • Query processing: generate results using indexed structures given the user query (online)

    • User interaction: Interface between user and search engine
    • Ranking: core component of search engine; outputs list of ranked documents
    • Evaluation: measure and monitor effectiveness and efficiency

Q: What are the main high-level components of an indexing procedure?

Q: What are the main high-level components of an query procedure?

Q: Why is the document data store connected to user interaction and not to ranking in the typical architecture of the query process?

A: It was somehow that snippets from the query are used to retrieve documents from the document data store, that then go into the ranking procedure. The ranking procedure is expensive, so perhaps this early step really helps limiting the size of the ranking operation. Notice that we have a bi-directional arrow "User Interaction" <-> "Ranking". The ranking algorithm works with the index and sends results to the user interface. But no raw results are shown to the user; they are piped through the UI. I didn't really get this based on the lecture, but it becomes clear in Croft-Metzler-Stroman-Search Engines Information Retrieval in Practice - Chapter 2

  • Query input: interface and parser for a query language
    • Most search engines are designed for keyword-based queries and do not function well for longer sentences
  • Query transformation
    • Similar to text transformation: tokenizing, stopping, stemming
    • Spell checking
    • Query suggestion (using query logs)
    • Query expansion
      • Relevance feedback: expansion "based on term occurrences in documents that are identified as relevant by the user." (25)
  • Present results, may include
    • Summarization/snippet generation
    • Highlighting of result based on query
    • Clustering output to find similar documents
    • Determine relevant advertising (BOOOOOOOH)

Q: Explain the arrows going in and out of "User Interaction" in a typical query process scheme.

So indeed we have the translating of the query into a query language that is used in the ranking procedure. This may also involve further processing such as query expansion. Also, we searched based on document representations (e.g. extracted features) rather than the whole document for efficiency. When displaying results to the user, we return (links to) documents but may need additional information from the document data store. We may generate snippets, we may summarize relevant documents, or cluster the output. These are all operations that require access to the document data store.