Maximum marginal likelihood estimation for nonnegative dictionary learning in the gamma-poisson model

Onur Dikmen*, Cédric Févotte

*Corresponding author for this work

    Research output: Contribution to journalArticleScientificpeer-review

    24 Citations (Scopus)

    Abstract

    In this paper we describe an alternative to standard nonnegative matrix factorization (NMF) for nonnegative dictionary learning, i.e., the task of learning a dictionary with nonnegative values from nonnegative data, under the assumption of nonnegative expansion coefficients. A popular cost function used for NMF is the Kullback-Leibler divergence, which underlies a Poisson observation model. NMF can thus be considered as maximization of the joint likelihood of the dictionary and the expansion coefficients. This approach lacks optimality because the number of parameters (which include the expansion coefficients) grows with the number of observations. In this paper we describe variational Bayes and Monte-Carlo EM algorithms for optimization of the marginal likelihood, i.e., the likelihood of the dictionary where the expansion coefficients have been integrated out (given a Gamma prior). We compare the output of both maximum joint likelihood estimation (i.e., standard NMF) and maximum marginal likelihood estimation (MMLE) on real and synthetical datasets. In particular we present face reconstruction results on CBCL dataset and text retrieval results over the musiXmatch dataset, a collection of word counts in song lyrics. The MMLE approach is shown to prevent overfitting by automatically pruning out irrelevant dictionary columns, i.e., embedding automatic model order selection.

    Original languageEnglish
    Article number6232464
    Pages (from-to)5163-5175
    Number of pages13
    JournalIEEE Transactions on Signal Processing
    Volume60
    Issue number10
    DOIs
    Publication statusPublished - 2012
    MoE publication typeA1 Journal article-refereed

    Keywords

    • Automatic relevance determination
    • model order selection
    • Monte Carlo EM
    • nonnegative matrix factorization
    • sparse coding
    • variational EM

    Fingerprint

    Dive into the research topics of 'Maximum marginal likelihood estimation for nonnegative dictionary learning in the gamma-poisson model'. Together they form a unique fingerprint.

    Cite this