Perception vs Reality: Measuring Machine Translation Post-Editing Productivity

This paper presents a study of user-perceived vs real machine translation (MT) post-editingeffort and productivity gains, focusing on two bidirectional language pairs: English—German and English—Dutch. Twenty experienced media professionals post-edited statisticalMT output and also manually translated comparative texts within a production environment.The paper compares the actual post-editing time against the users’ perception of theeffort and time required to post-edit the MT output to achieve publishable quality, thusmeasuring real (vs perceived) productivity gains. Although for all the language pairs usersperceived MT post-editing to be slower, in fact it proved to be a faster option than manualtranslation for two translation directions out of four, i.e. for Dutch→English, and (marginally)for English→German. For further objective scrutiny, the paper also checks the correlationof three state-of-the-art automatic MT evaluation metrics (BLEU, METEOR and TER) with the actual post-editing time.