The WMT'18 Morpheval test suites for English-Czech, English-German, English-Finnish and Turkish-English

Franck Burlot, Yves Scherrer, Vinit Ravishankar, Ondřej Bojar, Stig-Arne Grönroos, Maarit Koponen, Tommi Nieminen, François Yvon

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

65 Downloads (Pure)


Progress in the quality of machine translation output calls for new automatic evaluation procedures and metrics. In this paper, we extend the Morpheval protocol introduced by Burlot and Yvon (2017) for the English-toCzech and English-to-Latvian translation directions to three additional language pairs, and report its use to analyze the results of WMT 2018’s participants for these language pairs. Considering additional, typologically varied source and target languages also enables us to draw some generalizations regarding this morphology-oriented evaluation procedure
Original languageEnglish
Title of host publicationThird Conference on Machine Translation (WMT18); Brussels, Belgium
Number of pages15
ISBN (Electronic)978-1-948087-81-0
Publication statusPublished - 31 Oct 2018
MoE publication typeA4 Article in a conference publication
EventConference on Machine Translation - Brussels, Belgium
Duration: 31 Oct 20181 Dec 2018
Conference number: 3


ConferenceConference on Machine Translation
Abbreviated titleWMT


  • machine translation
  • morphology
  • evaluation

Cite this