BibSonomy :: publication ::

tag user group author concept BibTeX key search:all search:diego_ma

The blue social bookmark and publication sharing system.

entry of diego_ma:

(0)

This publication has not been reviewed yet.

rating distribution

average user rating

The average rating is computed over all reviews. However, some of them may be invisible to you due to the visibility setting chosen by the reviewers.

(0.0 of 5.0 based on 0 reviews)

In Vitro and In Vivo Evaluations of Parsing Systems Within the Context of Answer Extraction

by: Diego Mollá, and Ben Hutchinson

(2002) .

Abstract

A wide variety of parser and/or grammar evaluation methods have been reported in the literature. However, in most cases these evaluations take the parsers independently in vitro evaluations, and only in a few cases has the effect of different parsers in real applications been measured in vivo evaluations. This paper compares two evaluations of the Link Grammar parser and the Conexor Functional Dependency Grammar parser. The parsing systems, despite both being dependency-based, return different types of dependencies, making a direct comparison impossible. In the first evaluation, the accuracy of the parsers is compared in vitro by converting the dependencies into grammatical relations and using the methodology of Carroll:1998 for parser comparison. In the second evaluation, the parsers' impact in a practical application is compared in vivo within the context of answer extraction. The differences in the results are significant and raise questions on the usefulness of purely in vitro evaluations.

BibTeX record

@unpublished{Molla:anlp02:preparation,
  abstract = {A wide variety of parser and/or grammar evaluation methods have been reported in the literature. However, in most cases these evaluations take the parsers independently (\emph{in vitro} evaluations), and only in a few cases has the effect of different parsers in real applications been measured (\emph{in vivo} evaluations). This paper compares two evaluations of the Link Grammar parser and the Conexor Functional Dependency Grammar parser. The parsing systems, despite both being dependency-based, return different types of dependencies, making a direct comparison impossible. In the first evaluation, the accuracy of the parsers is compared \emph{in vitro} by converting the dependencies into grammatical relations and using the methodology of \newcite{Carroll:1998} for parser comparison. In the second evaluation, the parsers' impact in a practical application is compared \emph{in vivo} within the context of answer extraction. The differences in the results are significant and raise questions on the usefulness of purely \emph{in vitro} evaluations.},
  added-at = {2008-01-29T08:16:55.000+0100},
  author = {Moll{\'a}, Diego and Hutchinson, Ben},
  biburl = {http://www.bibsonomy.org/bibtex/2ddf167520fb423a651e0c5dcb062f1f2/diego_ma},
  interhash = {bd10546650953340c50de023e6c7c51e},
  intrahash = {ddf167520fb423a651e0c5dcb062f1f2},
  keywords = {AnswerFinder parsers evaluation gram_rels molla_publication},
  note = {In preparation},
  timestamp = {2008-01-29T08:16:55.000+0100},
  title = {In Vitro and In Vivo Evaluations of Parsing Systems Within the Context of Answer Extraction},
  year = 2002
}

Endnote record

%0 Unpublished Work
%1 Molla:anlp02:preparation
%A Moll{\'a}, Diego
%A Hutchinson, Ben
%D 2002
%K 
%T In Vitro and In Vivo Evaluations of Parsing Systems Within the Context of Answer Extraction
%X A wide variety of parser and/or grammar evaluation methods have been reported in the literature. However, in most cases these evaluations take the parsers independently (\emph{in vitro} evaluations), and only in a few cases has the effect of different parsers in real applications been measured (\emph{in vivo} evaluations). This paper compares two evaluations of the Link Grammar parser and the Conexor Functional Dependency Grammar parser. The parsing systems, despite both being dependency-based, return different types of dependencies, making a direct comparison impossible. In the first evaluation, the accuracy of the parsers is compared \emph{in vitro} by converting the dependencies into grammatical relations and using the methodology of \newcite{Carroll:1998} for parser comparison. In the second evaluation, the parsers' impact in a practical application is compared \emph{in vivo} within the context of answer extraction. The differences in the results are significant and raise questions on the usefulness of purely \emph{in vitro} evaluations.