5th Textual Entailment Challenge @ TAC 2009, Textual Entailment Search Pilot - Task Guidelines
Online, (2009)Abstract
During the RTE4 Challenge workshop at TAC 2008 the need to move towards more realistic scenarios was stressed once again, both by organizers and participants. In the campaigns held so far, the necessity of getting acquainted with the textual entailment task had motivated proposing only test sets in which the pairs were artificially adapted in order to facilitate the study of the different entailment phenomena; however, the progress that has been made, now allows to make a step forward and start to test RTE systems against real data. In order to meet this demand, a Textual Entailment Search Pilot task has been set up, aimed principally at: enumerate producing a data set which reflects the natural distribution of entailment in a corpus and presents all the problems that can arise while detecting textual entailment in a natural setting; analyzing the potential impact of textual entailment recognition on a real NLP application task, namely the Summarization task as proposed by the Summarization community in the 2008 Text Analysis Conference at NIST. enumerate This document provides a definition of the Pilot Search Task and a description of the data set, together with the instructions on how to take part in the exercise.