Comparative analysis of clicks and judgments ir evaluation

From WikiPapers
Jump to: navigation, search

Comparative analysis of clicks and judgments ir evaluation is a 2009 conference paper written in English by Kamps J., Koolen M., Trotman A. and published in Proceedings of Workshop on Web Search Click Data, WSCD'09.

[edit] Abstract

Queries and click-through data taken from search engine transaction logs is an attractive alternative to traditional test collections, due to its volume and the direct relation to end-user querying. The overall aim of this paper is to answer the question: How does click-through data differ from explicit human relevance judgments in information retrieval evaluation? We compare a traditional test collection with manual judgments to transaction log based test collections- by using queries as topics and subsequent clicks as pseudorelevance judgments for the clicked results. Specifically, we investigate the following two research questions: Firstly, are there significant differences between clicks and relevance judgments. Earlier research suggests that although clicks and explicit judgments show reasonable agreement, clicks are different from static absolute relevance judgments. Secondly, are there significant differences between system ranking based on clicks and based on relevance judgments? This is an open uestion, but earlier research suggests that comparative evaluation in terms of system ranking is remarkably robust. Copyright 2009 ACM.

[edit] References

This section requires expansion. Please, help!

Cited by

Probably, this publication is cited by others, but there are no articles available for them in WikiPapers. Cited 7 time(s)