This paper describes an approach to the diagnosticevaluation of machine translation (MT) based on linguistic checkpoints, which can provide valuable information both to the developers and to the end-users of MT systems.We present a flexible framework and a new tool, DELiC4MT, for fine-grained diagnostic MT evaluation which can be extended to anylanguage pair and applied to any evaluation target, once the phenomena of interest arecovered by the linguistic analysis. As a case study, we evaluate the CoSyne1 MT software against four leading web-based MT systems across a set of linguistic phenomena for three language pairs (from German, Italian and Dutch into English).
A Framework for Diagnostic Evaluation of Machine Translation Based on Linguistic Checkpoints
Federico Gaspari;
2011-01-01
Abstract
This paper describes an approach to the diagnosticevaluation of machine translation (MT) based on linguistic checkpoints, which can provide valuable information both to the developers and to the end-users of MT systems.We present a flexible framework and a new tool, DELiC4MT, for fine-grained diagnostic MT evaluation which can be extended to anylanguage pair and applied to any evaluation target, once the phenomena of interest arecovered by the linguistic analysis. As a case study, we evaluate the CoSyne1 MT software against four leading web-based MT systems across a set of linguistic phenomena for three language pairs (from German, Italian and Dutch into English).I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
