Diversity-based reranking

@IR

Information retrieval systems often make the assumption that the relevance of documents is independent of the relevance of other documents. Consider for example probability based retrieval models. In practice however, the relevance of an item does depend on the other items that are displayed in the ranked list. One clear example is where the top-k results are all highly relevant, but essentially discuss the same topic. If the second, third and fourth hit contain the same information as the first, they are not in fact that relevant to include in a ranking. This is one pragmatic reason for people to look into ways to re-rank such results, such that the results are more diverse. Even though this may hurt some performance measure, most user studies show this increases user satisfaction.

There are of course also more idealistic reasons to promote diversity. I'd say we are then more interested in fairness, but the relation between diversity and fairness in the literature is unclear to me as of now. Diversity seems to be a bit more narrowly defined and usually refers to decreasing redundancy/promoting novelty in search results. It may also be things like "topic diversity", "domain diversity", where this redundancy principle is applied to different levels.

Rather than re-ranking results from an existing system, one may also incorporate a diversity measure into existing information retrieval models. This is a slightly different (and I'd say more difficult) topic.

This MMR paper is seminal:

  • "The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries" by Carbonell and Goldstein.

MMR is user-tunable: "some users may prefer to drill down on a narrow topic; and others a panoramic sampling bearing relevance to the query."

Other papers:

  • "A Bayesian Learning Approach to Promoting Diversity in Ranking for Biomedical Information Retrieval" - Huang and Hu

  • "Diversity in recommender systems - a survey" by Kunaver and Pozrl.