Advances in Nonnegative Matrix Decomposition with Application to Cluster Analysis

Research output: ThesisDoctoral ThesisCollection of Articles

Researchers

  • He Zhang

Research units

Abstract

Nonnegative Matrix Factorization (NMF) has found a wide variety of applications in machine learning and data mining. NMF seeks to approximate a nonnegative data matrix by a product of several low-rank factorizing matrices, some of which are constrained to be nonnegative. Such additive nature often results in parts-based representation of the data, which is a desired property especially for cluster analysis. This thesis presents advances in NMF with application in cluster analysis. It reviews a class of higher-order NMF methods called Quadratic Nonnegative Matrix Factorization (QNMF). QNMF differs from most existing NMF methods in that some of its factorizing matrices occur twice in the approximation. The thesis also reviews a structural matrix decomposition method based on Data-Cluster-Data (DCD) random walk. DCD goes beyond matrix factorization and has a solid probabilistic interpretation by forming the approximation with cluster assigning probabilities only. Besides, the Kullback-Leibler divergence adopted by DCD is advantageous in handling sparse similarities for cluster analysis. Multiplicative update algorithms have been commonly used for optimizing NMF objectives, since they naturally maintain the nonnegativity constraint of the factorizing matrix and require no user-specified parameters. In this work, an adaptive multiplicative update algorithm is proposed to increase the convergence speed of QNMF objectives. Initialization conditions play a key role in cluster analysis. In this thesis, a comprehensive initialization strategy is proposed to improve the clustering performance by combining a set of base clustering methods. The proposed method can better accommodate clustering methods that need a careful initialization such as the DCD. The proposed methods have been tested on various real-world datasets, such as text documents, face images, protein, etc. In particular, the proposed approach has been applied to the cluster analysis of emotional data.

Details

Translated title of the contributionAdvances in Nonnegative Matrix Decomposition with Application to Cluster Analysis
Original languageEnglish
QualificationDoctor's degree
Awarding Institution
Supervisors/Advisors
Publisher
  • Aalto University
Print ISBNs978-952-60-5828-3
Electronic ISBNs978-952-60-5829-0
Publication statusPublished - 2014
MoE publication typeG5 Doctoral dissertation (article)

    Research areas

  • nonnegative matrix factorization, cluster analysis, multiplicative update rule, constrained optimization, initialization condition, image classification and retrieval, affective computing, image emotion

ID: 18355802