Machine translation into morphologically rich low-resource languages

Stig-Arne Grönroos

Machine translation into morphologically rich low-resource languages

Julkaisun otsikon käännös: Konekäännös morfologisesti rikkaisiin resurssiniukkoihin kieliin

Stig-Arne Grönroos

Speech Recognition

Tutkimustuotos: Doctoral Thesis › Collection of Articles

Abstrakti

Konekäännös on tärkeä luonnollisten kielten käsittelyn sovellus, joka mahdollistaa entistä laajemman pääsyn tietoon monikielisessä maailmassa, sekä edesauttaa kulttuurista vuorovaikutusta ja liiketoimintaa. Konekäännös on kehittynyt nopeasti viimeaikoina syviin neuroverkkoihin kohdistuvan tutkimuksen ansiosta. Etenkin käännöksen kielellinen sujuvuus on edistynyt. Koska menetelmät edellyttävät suuria datamääriä, kehitys on keskittynyt hyvin resursoiduille kielille. Tämän väitöskirjan tavoitteena on edistää konekäännöstä kun kohdekielenä on morfologisesti rikas kieli, jolle on saatavilla niukasti resursseja. Kielen rikas morfologia johtaa sananmuotojen määrän kombinatoriseen räjähdykseen tuottaen erittäin suuria sanastoja. Harvinaisia sananmuotoja on vaikea mallintaa. Väitöskirjassa näihin haasteisiin vastataan hyödyntäen useaa lähestymistapaa. Pääasiallinen lähestymistapa on sanojen pilkonta osiin. Pilkonnan avulla saadut esitystavat ovat helpompia mallintaa, minkä lisäksi pilkonnalla voidaan parantaa kielten välistä symmetriaa. On tärkeää hyödyntää resursseja läheisistä sovelluksista, esimerkiksi sukulaiskielten rinnakkaista tekstiä sisältävistä aineistoista, sekä yksikielisistä aineistoista. Väitöskirjassa käytetään kieliriippumattomia menetelmiä ja erilaisia koneoppimisasetelmia, kuten ohjaamatonta oppimista. Apuaineistoja hyödynnetään käyttämällä monikielistä oppimista, datan rikastamista,kohinaa poistavaa sekvenssiautoenkooderia, sekä pilkontaregularisointia. Aktiivista koneoppimista käytetään tehokkaampaan annotaatioiden keräämiseen, ja aikataulutettua monen tehtävän oppimista monikielisen oppimisen tehostamiseen. Väitöskirjassa esitellään viisi uutta menetelmää sanojen pilkontaan: Morfessor FlatCat, Omorfi-restricted Morfessor, Cognate Morfessor, Morfessor EM+Prune, sekä puoliohjattu neuroverkkoihin perustuva menetelmä. Morfessor FlatCat -menetelmälle esitellään aktiivisen koneoppimisen strategia. Pilkonnan laatua arvioidaan sekä suorilla että epäsuorilla automaattisilla evaluaatioilla.Morfessor EM+Prunen löytämillä malleilla on sekä alempi kustannusfunktion arvo että parempi pilkonnan laatu kuin Morfessor Baseline -menetelmällä. Aktiivinen koneoppiminen on satunnaista valintaa parempi annotaatoiden keräämiseen. Puoliohjatussa pilkonnassa paras laatu saavutetaan käyttämällä Morfessor FlatCatin pilkontoja piirteinä ehdollisessa satunnaiskentässä. Konekäännöksen menetelminä esitellään kohdekielen puolella tapahtuva monen tehtävän oppiminen sekä aikataulutettu monen tehtävän oppiminen, joka hyödyntää kohinaa poistavaa sekvenssiautoenkooderia. Lisäksi esitellään morfologisesti rikkaille kohdekielille soveltuva evaluaatiomenetelmä, LeBLEU. Konekäännöksen laatua arvioidaan sekä automaattisilla että ihmisarvioihin perustuvilla menetelmillä. Kun tavoitteena olevan käännöstehtävän resurssit ovat vähäisiä,tärkeimmät apuaineistot ovat sukulaiskielet. Myös yksikielisistä aineistoista on hyötyä.

Julkaisun otsikon käännös	Konekäännös morfologisesti rikkaisiin resurssiniukkoihin kieliin
Alkuperäiskieli	Englanti
Pätevyys	Tohtorintutkinto
Myöntävä instituutio	Aalto-yliopisto
Valvoja/neuvonantaja	Kurimo, Mikko, Vastuuprofessori Virpioja, Sami, Ohjaaja
Kustantaja	Aalto University
Painoksen ISBN	978-952-64-0168-3
Sähköinen ISBN	978-952-64-0169-0
Tila	Julkaistu - 2020
OKM-julkaisutyyppi	G5 Artikkeliväitöskirja

Tutkimusalat

konekäännös
morfeeemipilkonta
ohjaamaton oppiminen
puoliohjattu oppiminen
siirto-oppiminen
monen tehtävän oppiminen
aktiivinen koneoppiminen
syvät neuroverkot

Pääsy asiakirjaan

http://urn.fi/URN:ISBN:978-952-64-0169-0

Science-IT
Mikko Hakala (Manager)
Perustieteiden korkeakoulu
Laitteistot/tilat: Facility

Siteeraa tätä

@phdthesis{3e00c84b291d4eddb150c8fc1d565212,

title = "Machine translation into morphologically rich low-resource languages",

abstract = "Machine translation is an important natural language processing application, enabling widened access to information, cultural interchange, and business opportunities in a multilingual world. Driven by research into deep neural networks, machine translation has recently made rapid advances, particularly in the fluency of the translation output. As the methods tend to be data-hungry, high-resource languages have benefited more than low-resource ones. In this work, the aim is to improve machine translation into low-resource morphologically rich languages. Rich morphology leads to a combinatorial explosion in the number of word forms,resulting in very large vocabularies, containing many poorly modeled rare words. This thesis addresses these challenges with multiple approaches. The focus is on methods for segmenting words into subwords, to get more frequent and thus easier learned representations, and to increase the symmetry between languages. It is important to exploit additional resources from related tasks,such as parallel data from related high-resource language pairs and monolingual data from both low- and high-resource languages. Useful auxiliary data sets for multimodal translation can befound from captioning and text-only translation tasks. The methods for exploiting this auxiliary data include cross-lingual learning and data augmentation e.g. using denoising sequence autoen-coders and subword regularization. Learning setups used in the thesis include using unsupervised and language-independent methods, using active learning to guide an annotation effort to produce more informative data, and using scheduled multi-task learning to improve cross-lingual transfer. Contributions of the thesis include five novel segmentation methods: Morfessor FlatCat, Omorfi-restricted Morfessor, Cognate Morfessor, Morfessor EM+Prune, and a semi-supervised neural method. An active learning strategy for Morfessor FlatCat is presented. Evaluation of segmentation quality is performed using both intrinsic and extrinsic automatic methods. Morfessor EM+Prunefinds models with both lower cost and better quality in unsupervised segmentation than Morfessor Baseline. Active learning is superior to random selection for collecting annotations. The best performance in semi-supervised segmentation is achieved when using Morfessor FlatCat segmentations as features in a conditional random field. Contributions to machine translation include a target-side multi-task learning scheme, and scheduled multi-task learning with a denoising sequence autoencoder. LeBLEU, an evaluation measure suitable for morphologically rich languages is presented. Evaluation of translation quality is performed using both automatic and human evaluation. When resources are scarce, the most important auxiliary data comes from related languages. Other types of auxiliary data, such as monolingual corpora, are also beneficial and the gains are partly cumulative.",

keywords = "machine translation, morpheme segmentation, subwords, unsupervised learning, semi-supervised learning, transfer learning, multi-task learning, active learning, konek{\"a}{\"a}nn{\"o}s, morfeeemipilkonta, ohjaamaton oppiminen, puoliohjattu oppiminen, siirto-oppiminen, monen teht{\"a}v{\"a}n oppiminen, aktiivinen koneoppiminen, syv{\"a}t neuroverkot, machine translation, morpheme segmentation, subwords, unsupervised learning, semi-supervised learning, transfer learning, multi-task learning, active learning",

author = "Stig-Arne Gr{\"o}nroos",

year = "2020",

language = "English",

isbn = "978-952-64-0168-3",

series = "Aalto University publication series DOCTORAL DISSERTATIONS",

publisher = "Aalto University",

number = "202",

address = "Finland",

school = "Aalto University",

}

TY - BOOK

T1 - Machine translation into morphologically rich low-resource languages

AU - Grönroos, Stig-Arne

PY - 2020

Y1 - 2020

N2 - Machine translation is an important natural language processing application, enabling widened access to information, cultural interchange, and business opportunities in a multilingual world. Driven by research into deep neural networks, machine translation has recently made rapid advances, particularly in the fluency of the translation output. As the methods tend to be data-hungry, high-resource languages have benefited more than low-resource ones. In this work, the aim is to improve machine translation into low-resource morphologically rich languages. Rich morphology leads to a combinatorial explosion in the number of word forms,resulting in very large vocabularies, containing many poorly modeled rare words. This thesis addresses these challenges with multiple approaches. The focus is on methods for segmenting words into subwords, to get more frequent and thus easier learned representations, and to increase the symmetry between languages. It is important to exploit additional resources from related tasks,such as parallel data from related high-resource language pairs and monolingual data from both low- and high-resource languages. Useful auxiliary data sets for multimodal translation can befound from captioning and text-only translation tasks. The methods for exploiting this auxiliary data include cross-lingual learning and data augmentation e.g. using denoising sequence autoen-coders and subword regularization. Learning setups used in the thesis include using unsupervised and language-independent methods, using active learning to guide an annotation effort to produce more informative data, and using scheduled multi-task learning to improve cross-lingual transfer. Contributions of the thesis include five novel segmentation methods: Morfessor FlatCat, Omorfi-restricted Morfessor, Cognate Morfessor, Morfessor EM+Prune, and a semi-supervised neural method. An active learning strategy for Morfessor FlatCat is presented. Evaluation of segmentation quality is performed using both intrinsic and extrinsic automatic methods. Morfessor EM+Prunefinds models with both lower cost and better quality in unsupervised segmentation than Morfessor Baseline. Active learning is superior to random selection for collecting annotations. The best performance in semi-supervised segmentation is achieved when using Morfessor FlatCat segmentations as features in a conditional random field. Contributions to machine translation include a target-side multi-task learning scheme, and scheduled multi-task learning with a denoising sequence autoencoder. LeBLEU, an evaluation measure suitable for morphologically rich languages is presented. Evaluation of translation quality is performed using both automatic and human evaluation. When resources are scarce, the most important auxiliary data comes from related languages. Other types of auxiliary data, such as monolingual corpora, are also beneficial and the gains are partly cumulative.

AB - Machine translation is an important natural language processing application, enabling widened access to information, cultural interchange, and business opportunities in a multilingual world. Driven by research into deep neural networks, machine translation has recently made rapid advances, particularly in the fluency of the translation output. As the methods tend to be data-hungry, high-resource languages have benefited more than low-resource ones. In this work, the aim is to improve machine translation into low-resource morphologically rich languages. Rich morphology leads to a combinatorial explosion in the number of word forms,resulting in very large vocabularies, containing many poorly modeled rare words. This thesis addresses these challenges with multiple approaches. The focus is on methods for segmenting words into subwords, to get more frequent and thus easier learned representations, and to increase the symmetry between languages. It is important to exploit additional resources from related tasks,such as parallel data from related high-resource language pairs and monolingual data from both low- and high-resource languages. Useful auxiliary data sets for multimodal translation can befound from captioning and text-only translation tasks. The methods for exploiting this auxiliary data include cross-lingual learning and data augmentation e.g. using denoising sequence autoen-coders and subword regularization. Learning setups used in the thesis include using unsupervised and language-independent methods, using active learning to guide an annotation effort to produce more informative data, and using scheduled multi-task learning to improve cross-lingual transfer. Contributions of the thesis include five novel segmentation methods: Morfessor FlatCat, Omorfi-restricted Morfessor, Cognate Morfessor, Morfessor EM+Prune, and a semi-supervised neural method. An active learning strategy for Morfessor FlatCat is presented. Evaluation of segmentation quality is performed using both intrinsic and extrinsic automatic methods. Morfessor EM+Prunefinds models with both lower cost and better quality in unsupervised segmentation than Morfessor Baseline. Active learning is superior to random selection for collecting annotations. The best performance in semi-supervised segmentation is achieved when using Morfessor FlatCat segmentations as features in a conditional random field. Contributions to machine translation include a target-side multi-task learning scheme, and scheduled multi-task learning with a denoising sequence autoencoder. LeBLEU, an evaluation measure suitable for morphologically rich languages is presented. Evaluation of translation quality is performed using both automatic and human evaluation. When resources are scarce, the most important auxiliary data comes from related languages. Other types of auxiliary data, such as monolingual corpora, are also beneficial and the gains are partly cumulative.

KW - machine translation

KW - morpheme segmentation

KW - subwords

KW - unsupervised learning

KW - semi-supervised learning

KW - transfer learning

KW - multi-task learning

KW - active learning

KW - konekäännös

KW - morfeeemipilkonta

KW - ohjaamaton oppiminen

KW - puoliohjattu oppiminen

KW - siirto-oppiminen

KW - monen tehtävän oppiminen

KW - aktiivinen koneoppiminen

KW - syvät neuroverkot

KW - machine translation

KW - morpheme segmentation

KW - subwords

KW - unsupervised learning

KW - semi-supervised learning

KW - transfer learning

KW - multi-task learning

KW - active learning

M3 - Doctoral Thesis

SN - 978-952-64-0168-3

T3 - Aalto University publication series DOCTORAL DISSERTATIONS

PB - Aalto University

ER -

Machine translation into morphologically rich low-resource languages

Abstrakti

Tutkimusalat

Pääsy asiakirjaan

Sormenjälki

Laitteet

Science-IT

Siteeraa tätä