Interpretable Word Embeddings via Informative Priors

Miriam Hurtado Bodell; Martin Arvidsson; Måns Magnusson

Interpretable Word Embeddings via Informative Priors

Miriam Hurtado Bodell, Martin Arvidsson, Måns Magnusson

Linköping University

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

5 Citations (Scopus)

Abstract

Word embeddings have demonstrated strong performance on NLP tasks. However, lack of interpretability and the unsupervised nature of word embeddings have limited their use within computational social science and digital humanities. We propose the use of informative priors to create interpretable and domain-informed dimensions for probabilistic word embeddings. Experimental results show that sensible priors can capture latent semantic concepts better than or on-par with the current state of the art, while retaining the simplicity and generalizability of using priors.

Original language	English
Title of host publication	The 2019 Conference on Empirical Methods in Natural Language Processing And the 9th International Joint Conference on Natural Language Processing
Subtitle of host publication	Proceedings of System Demonstrations
Publisher	Association for Computational Linguistics
Pages	6324-6330
ISBN (Print)	978-1-950737-92-5
Publication status	Published - 2019
MoE publication type	A4 Conference publication
Event	Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing - Hong Kong, China Duration: 3 Nov 2019 → 7 Nov 2019

Conference

Conference	Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing
Abbreviated title	EMNLP/IJCNLP
Country/Territory	China
City	Hong Kong
Period	03/11/2019 → 07/11/2019

Access to Document

https://arxiv.org/abs/1909.01459

Cite this

Bodell, M. H., Arvidsson, M., & Magnusson, M. (2019). Interpretable Word Embeddings via Informative Priors. In The 2019 Conference on Empirical Methods in Natural Language Processing And the 9th International Joint Conference on Natural Language Processing: Proceedings of System Demonstrations (pp. 6324-6330). Association for Computational Linguistics. https://arxiv.org/abs/1909.01459

@inproceedings{b47f20a06abb4a4cab2b1f7c35c14f9e,

title = "Interpretable Word Embeddings via Informative Priors",

abstract = "Word embeddings have demonstrated strong performance on NLP tasks. However, lack of interpretability and the unsupervised nature of word embeddings have limited their use within computational social science and digital humanities. We propose the use of informative priors to create interpretable and domain-informed dimensions for probabilistic word embeddings. Experimental results show that sensible priors can capture latent semantic concepts better than or on-par with the current state of the art, while retaining the simplicity and generalizability of using priors.",

author = "Bodell, {Miriam Hurtado} and Martin Arvidsson and M{\aa}ns Magnusson",

year = "2019",

language = "English",

isbn = "978-1-950737-92-5",

pages = "6324--6330",

booktitle = "The 2019 Conference on Empirical Methods in Natural Language Processing And the 9th International Joint Conference on Natural Language Processing",

publisher = "Association for Computational Linguistics",

address = "United States",

note = "Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP/IJCNLP ; Conference date: 03-11-2019 Through 07-11-2019",

}

Bodell, MH, Arvidsson, M & Magnusson, M 2019, Interpretable Word Embeddings via Informative Priors. in The 2019 Conference on Empirical Methods in Natural Language Processing And the 9th International Joint Conference on Natural Language Processing: Proceedings of System Demonstrations. Association for Computational Linguistics, pp. 6324-6330, Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China, 03/11/2019. <https://arxiv.org/abs/1909.01459>

Interpretable Word Embeddings via Informative Priors. / Bodell, Miriam Hurtado; Arvidsson, Martin; Magnusson, Måns.
The 2019 Conference on Empirical Methods in Natural Language Processing And the 9th International Joint Conference on Natural Language Processing: Proceedings of System Demonstrations. Association for Computational Linguistics, 2019. p. 6324-6330.

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

TY - GEN

T1 - Interpretable Word Embeddings via Informative Priors

AU - Bodell, Miriam Hurtado

AU - Arvidsson, Martin

AU - Magnusson, Måns

PY - 2019

Y1 - 2019

N2 - Word embeddings have demonstrated strong performance on NLP tasks. However, lack of interpretability and the unsupervised nature of word embeddings have limited their use within computational social science and digital humanities. We propose the use of informative priors to create interpretable and domain-informed dimensions for probabilistic word embeddings. Experimental results show that sensible priors can capture latent semantic concepts better than or on-par with the current state of the art, while retaining the simplicity and generalizability of using priors.

AB - Word embeddings have demonstrated strong performance on NLP tasks. However, lack of interpretability and the unsupervised nature of word embeddings have limited their use within computational social science and digital humanities. We propose the use of informative priors to create interpretable and domain-informed dimensions for probabilistic word embeddings. Experimental results show that sensible priors can capture latent semantic concepts better than or on-par with the current state of the art, while retaining the simplicity and generalizability of using priors.

M3 - Conference article in proceedings

SN - 978-1-950737-92-5

SP - 6324

EP - 6330

BT - The 2019 Conference on Empirical Methods in Natural Language Processing And the 9th International Joint Conference on Natural Language Processing

PB - Association for Computational Linguistics

T2 - Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing

Y2 - 3 November 2019 through 7 November 2019

ER -

Interpretable Word Embeddings via Informative Priors

Abstract

Conference

Access to Document

Fingerprint

Cite this