FinnPos: an open-source morphological tagging and lemmatization toolkit for Finnish

Research output: Contribution to journalArticleScientificpeer-review

Researchers

  • Miikka Silfverberg
  • Teemu Ruokolainen
  • Krister Lindén
  • Mikko Kurimo

Research units

  • University of Helsinki

Abstract

This paper describes FinnPos, an open-source morphological tagging and lemmatization toolkit for Finnish. The morphological tagging model is based on the averaged structured perceptron classifier. Given training data, new taggers are estimated in a computationally efficient manner using a combination of beam search and model cascade. The lemmatization is performed employing a combination of a rule-based morphological analyzer, OMorFi, and a data-driven lemmatization model. The toolkit is readily applicable for tagging and lemmatization of running text with models learned from the recently published Finnish Turku Dependency Treebank and FinnTreeBank. Empirical evaluation on these corpora shows that FinnPos performs favorably compared to reference systems in terms of tagging and lemmatization accuracy. In addition, we demonstrate that our system is highly competitive with regard to computational efficiency of learning new models and assigning analyses to novel sentences.

Details

Original languageEnglish
Pages (from-to)863-878
Number of pages16
JournalLANGUAGE RESOURCES AND EVALUATION
Volume50
Issue number4
Publication statusPublished - 1 Dec 2016
MoE publication typeA1 Journal article-refereed

    Research areas

  • Averaged perceptron, Data-driven lemmatization, Finnish, Morphological tagging, Open-source

ID: 9632963