This paper reports MT evaluation experimentsthat were conducted at the endof year 1 of the EU-funded CoSyne1 project for three language combinations,considering translations from German, Italian and Dutch into English. We present a comparative evaluation of the MT software developed within the project against four of the leading free web-basedMT systems across a range ofstate-of-the-art automatic evaluation metrics. The data sets from the news domainthat were created and used for training purposes and also for this evaluation exercise,which are available to the research community, are also described. The evaluation results for the news domainare very encouraging: the CoSyne MT software consistently beats the rule-based MT systems, and for translations from Italian and Dutch into English in particular the scores given by some of the standardautomatic evaluation metrics are not too distant from those obtained by wellestablishedstatistical online MT systems.
A Comparative Evaluation of Research vs. Online Machine Translation Systems
Federico Gaspari;
2011-01-01
Abstract
This paper reports MT evaluation experimentsthat were conducted at the endof year 1 of the EU-funded CoSyne1 project for three language combinations,considering translations from German, Italian and Dutch into English. We present a comparative evaluation of the MT software developed within the project against four of the leading free web-basedMT systems across a range ofstate-of-the-art automatic evaluation metrics. The data sets from the news domainthat were created and used for training purposes and also for this evaluation exercise,which are available to the research community, are also described. The evaluation results for the news domainare very encouraging: the CoSyne MT software consistently beats the rule-based MT systems, and for translations from Italian and Dutch into English in particular the scores given by some of the standardautomatic evaluation metrics are not too distant from those obtained by wellestablishedstatistical online MT systems.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
