FinnPos: an open-source morphological tagging and lemmatization toolkit for Finnish

Miikka Silfverberg*, Teemu Ruokolainen, Krister Lindén, Mikko Kurimo

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

10 Citations (Scopus)

Abstract

This paper describes FinnPos, an open-source morphological tagging and lemmatization toolkit for Finnish. The morphological tagging model is based on the averaged structured perceptron classifier. Given training data, new taggers are estimated in a computationally efficient manner using a combination of beam search and model cascade. The lemmatization is performed employing a combination of a rule-based morphological analyzer, OMorFi, and a data-driven lemmatization model. The toolkit is readily applicable for tagging and lemmatization of running text with models learned from the recently published Finnish Turku Dependency Treebank and FinnTreeBank. Empirical evaluation on these corpora shows that FinnPos performs favorably compared to reference systems in terms of tagging and lemmatization accuracy. In addition, we demonstrate that our system is highly competitive with regard to computational efficiency of learning new models and assigning analyses to novel sentences.

Original languageEnglish
Pages (from-to)863-878
Number of pages16
JournalLANGUAGE RESOURCES AND EVALUATION
Volume50
Issue number4
DOIs
Publication statusPublished - 1 Dec 2016
MoE publication typeA1 Journal article-refereed

Keywords

  • Averaged perceptron
  • Data-driven lemmatization
  • Finnish
  • Morphological tagging
  • Open-source

Fingerprint

Dive into the research topics of 'FinnPos: an open-source morphological tagging and lemmatization toolkit for Finnish'. Together they form a unique fingerprint.

Cite this