Spoken Dialogue Systems Evaluation
Evaluation of Text and Speech Systems, chapter 7, (2007)DOI: 10.1007/978-1-4020-5817-2_7
Abstract
This chapter first provides a brief introduction to evaluation methods and criteria and then presents two very different spoken dialogue research prototype systems and their evaluation. The first prototype is the non-task-oriented, multimodal Hans Christian Andersen (HCA) system for edutainment, the second prototype is the task-oriented, multimodal SENECA onboard system in the car. The systems were tested with representative users in the laboratory and in the field, respectively. For both systems we describe rationale for the chosen evaluation method, evaluation process, evaluation criteria, and evaluation results.