A framework for benchmarking entity-annotation systems
|A framework for benchmarking entity-annotation systems|
|Author(s)||Cornolti M., Ferragina P., Ciaramita M.|
|Published in||WWW 2013 - Proceedings of the 22nd International Conference on World Wide Web|
|Keyword(s)||Benchmark framework, Entity annotation, Wikipedia (Extra: Comparative evaluations, Design and implements, Entity annotation, Evaluation measures, Systems performance, Wikipedia, Benchmarking, World Wide Web, Data processing)|
|Article||BASE, CiteSeerX, Google Scholar|
|Web||Ask, Bing, Google (PDF), Yahoo!|
|Download and mirrors|
|Local copy||Not available|
|Remote mirror(s)||Not available|
|Export and share|
|BibTeX, CSV, RDF, JSON|
|Browse properties · List of conference papers|
A framework for benchmarking entity-annotation systems is a 2013 conference paper written in English by Cornolti M., Ferragina P., Ciaramita M. and published in WWW 2013 - Proceedings of the 22nd International Conference on World Wide Web.
In this paper we design and implement a benchmarking framework for fair and exhaustive comparison of entity-annotation systems. The framework is based upon the definition of a set of problems related to the entity-annotation task, a set of measures to evaluate systems performance, and a systematic comparative evaluation involving all publicly available datasets, containing texts of various types such as news, tweets and Web pages. Our framework is easily-extensible with novel entity annotators, datasets and evaluation measures for comparing systems, and it has been released to the public as open source1. We use this framework to perform the first extensive comparison among all available entity annotators over all available datasets, and draw many interesting conclusions upon their efficiency and effectiveness. We also draw conclusions between academic versus commercial annotators. Copyright is held by the International World Wide Web Conference Committee (IW3C2).
- This section requires expansion. Please, help!
Probably, this publication is cited by others, but there are no articles available for them in WikiPapers. Cited 5 time(s)