Overview of ResPubliQA 2009: Question answering evaluation over European legislation
|Overview of ResPubliQA 2009: Question answering evaluation over European legislation|
|Author(s)||Penas A., Forner P., Sutcliffe R., Rodrigo A., Forascu C., Alegria I., Giampiccolo D., Moreau N., Osenova P.|
|Published in||Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)|
|Keyword(s)||Unknown (Extra: Cross-language evaluation forums, European legislation, Evaluation measures, Information need, Natural language questions, QA system, Question Answering, Question-answering evaluation, Realistic scenario, System output, Test sets, Validation technologies, Wikipedia, Linguistics, Natural language processing systems, Information retrieval)|
|Article||BASE, CiteSeerX, Google Scholar|
|Web||Ask, Bing, Google (PDF), Yahoo!|
|Download and mirrors|
|Local copy||Not available|
|Remote mirror(s)||Not available|
|Export and share|
|BibTeX, CSV, RDF, JSON|
|Browse properties · List of conference papers|
Overview of ResPubliQA 2009: Question answering evaluation over European legislation is a 2010 conference paper written in English by Penas A., Forner P., Sutcliffe R., Rodrigo A., Forascu C., Alegria I., Giampiccolo D., Moreau N., Osenova P. and published in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics).
This paper describes the first round of ResPubliQA, a Question Answering (QA) evaluation task over European legislation, proposed at the Cross Language Evaluation Forum (CLEF) 2009. The exercise consists of extracting a relevant paragraph of text that satisfies completely the information need expressed by a natural language question. The general goals of this exercise are (i) to study if the current QA technologies tuned for newswire collections and Wikipedia can be adapted to a new domain (law in this case); (ii) to move to a more realistic scenario, considering people close to law as users, and paragraphs as system output; (iii) to compare current QA technologies with pure Information Retrieval (IR) approaches; and (iv) to introduce in QA systems the Answer Validation technologies developed in the past three years. The paper describes the task in more detail, presenting the different types of questions, the methodology for the creation of the test sets and the new evaluation measure, and analyzing the results obtained by systems and the more successful approaches. Eleven groups participated with 28 runs. In addition, we evaluated 16 baseline runs (2 per language) based only in pure IR approach, for comparison purposes. Considering accuracy, scores were generally higher than in previous QA campaigns.
- This section requires expansion. Please, help!
Probably, this publication is cited by others, but there are no articles available for them in WikiPapers. Cited 4 time(s)