Quiz for Information Retrieval course, 2020-2021 @Radboud

Questions collected from the lecture notes.

W1: Introduction

  • Q: Are queries good representations of user needs? If yes, why? If not, give an argument why queries and user needs would easily diverge.
  • Q: What are some of the ways in which we can close the gap between a query and a user's information need?
  • Q: What is the difference between effectiveness and efficiency?
  • Q: What are the main high-level components of an indexing procedure?
  • Q: What are the main high-level components of an query procedure?
  • Q: Why is the document data store connected to user interaction and not to ranking in the typical architecture of the query process?
  • Q: Explain the arrows going in and out of "User Interaction" in a typical query process scheme.

W2: Retrieval Models

  • Q: What is an information retrieval model?
  • Q: Why is it hard (or even: impossible) to prove the validity of an information retrieval model?
  • Q: What's the basic relevance assumption of a Vector Space Model?
  • Q: Explain the rationale behind the TF.IDF weighing scheme.
  • Q: Why is a logarithm used in the DF part of the TF.IDF formula?
  • Q: Which type of document representation is used in a Vector Space Model?
  • Q: What is the difference between the main relevance assumption of a probabilistic model versus a vector space model?
  • Q: What's the main relevance assumption of probabilistic retrieval models?
  • Q: Can you explain the rationale of using the transferred term frequency (e.g. in BM25) instead of regular term frequency? And what role does hyperparameter k play here?
  • Q: Is BM25 a vector space model or a probabilistic model?
  • Q: What is the rationale behind using document length normalization (for example in BM25)? And what role does hyperpameter b play?
  • Q: Can you identify the individual components of the BM25 retrieval model? Can you explain their rationale?
  • Q: What's the main assumption behind the query likelihood language model?
  • Q: Name two main high-level types of retrieval model.
  • Q: What is a language model?
  • Q: What's the difference between BM25 and a typical language modeling approach?
  • Q: What's the main difference between a mixed language modelling approach to fielded retrieval, rather than a "meta-search" approach?

W3: Retrieval Models II

  • Q: Why can one say that Dirichlet Smoothing reward shorter documents?
  • Q: In what way does the Query Likelihood model inherently include TF-IDF weighting and document length normalization?
  • Q: Does BM25F (fielded) usually also have a IDF-like term?

W4: Inverted indices

  • Q: What is a major difference between the datasets used for IR research and real-world datasets?
  • Q: What is a issue with lemmatization? Is this issue problematic in practice?
  • Q: Why do we not store postings in a binary matrix?
  • Q: Why is the postings list usually kept in sorting order?
  • Q: Why do we not use pointers to DocIDs for a posting?
  • Q: What is the different between bit-aligned and byte-aligned compression methods?
  • Q: How does Vbyte encoding aids compression / storage reduction of the inverted index?
  • Q: What's the advantage and downside of Vbyte compared to compression algorithms such as bzip and gzip?
  • Q: Why is gap encoding space-efficient in combination with VByte encoding?
  • Q: How can knowing the document frequency of a term potentially speed up query processing?
  • Q: Explain the difference and the trade-offs between document-at-a-time and term-at-a-time query processing.
  • Q: What is a typical data structure that is used in document-at-a-time query processing?

W5: Cranfield Paradigm

  • Q: Explain the two key points of the Cranfield paradigm and why they are useful for evaluation.
  • Q: What is "pooling" during the creation of test datasets (e.g. by TREC) and what problem does it solve?
  • Q: Explain the idea behind interpolated precision as used in precision-recall plots.
  • Q: Why are precision-recall plots so handy for comparing search engines? (Hint: a search engine A may be better than B in some settings but not in others)
  • Q: What is the main use case for Mean Reciprocal Rank (MMR)?
  • Q What is a main theoretical issue with MMR?
  • Q: Why is a logarithm used in the denominator of the cumulative gain?
  • Q: Why do we need to normalize the cumulative gain of a ranking for meaningful evaluation?
  • Q: How does NDCG factor in query difficulty?
  • Q: What are the two main assumptions employed in NDCG?

W6: Entity Retrieval and Linking

  • Q: What's a knowledge graph?
  • Q: How does entity retrieval relate to semantic search?
  • Q: When doing entity retrieval on movies, do you think this is an example of unstructured, semi-structured or structured input data?
  • Q: How would an inverted index for entity retrieval look like?
  • Q: What is the RDF format?
  • Q: Which retrieval models are suitable to be used for entity retrieval?
  • Q: What is predicate folding?
  • Q: Let's say we use BM25F, why do we then need predicate folding?
  • Q: What's Entity Linking?
  • Q: How does entity linking relate to entity retrieval?
  • Q: What are applications of entity linking?
  • Q: Is precision or recall more important during mention detection? Why?
  • Q: Which aspects of Wikipedia can be used during mention detection?
  • Q: What's a name dictionary in mention detection?
  • Q: Name two methods for ranking mentions during entity selection
  • Q: Explain the commonness measure used during entity selection

W7: Relevance Feedback

  • Q: Can you distinguish global and local query refinement?
  • Q: What does global query refinement usually do to precision and recall?
  • Q: Is query expansion a global or a local approach to query refinement?
  • Q: What is query drift and for what is it a problem?
  • Q: What is a downside of pseudo-relevance feedback?
  • Q: How can relevance models be seen as a process of query expansion and smoothing?

W8: Learning to Rank

  • Q: Why do we use the logistic function when expressing the probability of relevance in terms of a linear combination of (positively valued) features?
  • Q: What are three categories of learning to rank algorithms?

W9: Federated Search

  • Q: Why are standards so important for federated search?
  • Q: What are some open problems for Federated Search, relating to standards?
  • Q: What's the difference between Federated Search and Universal Search?
  • Q: Explain GLOSS
  • Q: Does GLOSS assume a cooperative or uncooperative environment?
  • Q: What role does database size play in GLOSS?
  • Q: In which way is it possible for individual search engines to cheat GLOSS in order to route more traffic to themselves?
  • Q: What's the difference between cooperative and uncooperative environments for federated search?
  • Q: Do query-based samples resemble a full index?
  • Q: Why does the "capture-recapture" database size estimation strategy of Broder et al. not work well for large search engines?
  • Q: Explain the sample-resample alternative to capture-recapture; to what extent does it work in an uncooperative environment?
  • Q: What's a clever trick to adjust sample-resample so that we do know the true document frequency for a query term without asking the resource?
  • Q: What's the importance of collection size C in ReDDE?

W10: Spoken Content Retrieval

  • Q: What is the difference between SCR and SDR?
  • Q: What's the difference between Spoken Term Detection (STD) and Spoken Content Retrieval (SCR).
  • Q: What are ways of dealing with noise and errors in ASR?
  • Q: What are the typical components of a HMM-based ASR system?

W11: Link Analysis

  • Q: In which way does web search have different information needs than traditional IR?
  • Q: What are the two main intuitions behind link analysis?
  • Q: Which main assumption of link analysis (and which 'link information') does PageRank emphasize?
  • Q: What's the purpose of citation smoothing in PageRank?
  • Q: Does PageRank take queries into account? Q: What's the role of hyperparameter alpha in the PageRank computation?

W12: NN for IR (dealing with data sparsity)

N.A.

W13: Conversational Agents

  • Q: What's sparse retrieval?
  • Q Are conversational agents necessarily speech based?
  • Q: What's the difference between conversational information seeking and interactive retrieval?

W14: Unbiased estimation using click data

  • Q: Explain different forms of bias that play a role in clickthrough data.
  • Q: What's the golden triangle in presentation bias?
  • Q: What are the two main approach to unbiased learning to rank?
  • Q: How do you deal with bias in an online approach to learning to rank?
  • Q: What's the difference between noise and bias?
  • Q: Why is averaging over many click results not effective against bias?
  • Q: Explain why bias leads/ can lead to self-reinforcing learning behavior (logging-policy-confirming behavior)