Comparative analysis of clicks and judgments ir evaluation
|Comparative analysis of clicks and judgments ir evaluation|
|Author(s)||Kamps J., Koolen M., Trotman A.|
|Published in||Proceedings of Workshop on Web Search Click Data, WSCD'09|
|Keyword(s)||Transaction log analysis, Web information retrieval, Wikipedia (Extra: Clickthrough data, Comparative analysis, Comparative evaluations, End users, Relevance judgment, Research questions, System rankings, Test Collection, Transaction log, Transaction log analysis, Web information retrieval, Wikipedia, Image retrieval, Information services, Online searching, Research, Search engines, World Wide Web)|
|Article||BASE, CiteSeerX, Google Scholar|
|Web||Ask, Bing, Google (PDF), Yahoo!|
|Download and mirrors|
|Local copy||Not available|
|Remote mirror(s)||Not available|
|Export and share|
|BibTeX, CSV, RDF, JSON|
|Browse properties · List of conference papers|
Comparative analysis of clicks and judgments ir evaluation is a 2009 conference paper written in English by Kamps J., Koolen M., Trotman A. and published in Proceedings of Workshop on Web Search Click Data, WSCD'09.
Queries and click-through data taken from search engine transaction logs is an attractive alternative to traditional test collections, due to its volume and the direct relation to end-user querying. The overall aim of this paper is to answer the question: How does click-through data differ from explicit human relevance judgments in information retrieval evaluation? We compare a traditional test collection with manual judgments to transaction log based test collections- by using queries as topics and subsequent clicks as pseudorelevance judgments for the clicked results. Specifically, we investigate the following two research questions: Firstly, are there significant differences between clicks and relevance judgments. Earlier research suggests that although clicks and explicit judgments show reasonable agreement, clicks are different from static absolute relevance judgments. Secondly, are there significant differences between system ranking based on clicks and based on relevance judgments? This is an open uestion, but earlier research suggests that comparative evaluation in terms of system ranking is remarkably robust. Copyright 2009 ACM.
- This section requires expansion. Please, help!
Probably, this publication is cited by others, but there are no articles available for them in WikiPapers. Cited 7 time(s)