DOLDA: a regularized supervised topic model for high-dimensional multi-class regression

Måns Magnusson*, Leif Jonsson, Mattias Villani

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

3 Citations (Scopus)
98 Downloads (Pure)

Abstract

Generating user interpretable multi-class predictions in data-rich environments with many classes and explanatory covariates is a daunting task. We introduce Diagonal Orthant Latent Dirichlet Allocation (DOLDA), a supervised topic model for multi-class classification that can handle many classes as well as many covariates. To handle many classes we use the recently proposed Diagonal Orthant probit model (Johndrow et al., in: Proceedings of the sixteenth international conference on artificial intelligence and statistics, 2013) together with an efficient Horseshoe prior for variable selection/shrinkage (Carvalho et al. in Biometrika 97:465–480, 2010). We propose a computationally efficient parallel Gibbs sampler for the new model. An important advantage of DOLDA is that learned topics are directly connected to individual classes without the need for a reference class. We evaluate the model’s predictive accuracy and scalability, and demonstrate DOLDA’s advantage in interpreting the generated predictions.

Original languageEnglish
Pages (from-to)175-201
Number of pages27
JournalComputational Statistics
Volume35
Issue number1
DOIs
Publication statusPublished - 1 Mar 2020
MoE publication typeA1 Journal article-refereed

Keywords

  • Diagonal Orthant probit model
  • Horseshoe prior
  • Interpretable models
  • Latent Dirichlet Allocation
  • Text classification

Fingerprint

Dive into the research topics of 'DOLDA: a regularized supervised topic model for high-dimensional multi-class regression'. Together they form a unique fingerprint.

Cite this