A high-performance implementation of Bayesian matrix factorization with limited communication

Tom Vander Aa; Xiangju Qin; Paul Blomstedt; Roel Wuyts; Wilfried Verachtert; Samuel Kaski

doi:10.1007/978-3-030-50433-5_1

A high-performance implementation of Bayesian matrix factorization with limited communication

Tom Vander Aa^*, Xiangju Qin, Paul Blomstedt, Roel Wuyts, Wilfried Verachtert, Samuel Kaski

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

1 Citation (Scopus)

Abstract

Matrix factorization is a very common machine learning technique in recommender systems. Bayesian Matrix Factorization (BMF) algorithms would be attractive because of their ability to quantify uncertainty in their predictions and avoid over-fitting, combined with high prediction accuracy. However, they have not been widely used on large-scale data because of their prohibitive computational cost. In recent work, efforts have been made to reduce the cost, both by improving the scalability of the BMF algorithm as well as its implementation, but so far mainly separately. In this paper we show that the state-of-the-art of both approaches to scalability can be combined. We combine the recent highly-scalable Posterior Propagation algorithm for BMF, which parallelizes computation of blocks of the matrix, with a distributed BMF implementation that users asynchronous communication within each block. We show that the combination of the two methods gives substantial improvements in the scalability of BMF on web-scale datasets, when the goal is to reduce the wall-clock time.

Original language	English
Title of host publication	Computational Science – ICCS 2020 - 20th International Conference, Proceedings
Editors	Valeria V. Krzhizhanovskaya, Gábor Závodszky, Michael H. Lees, Peter M.A. Sloot, Peter M.A. Sloot, Peter M.A. Sloot, Jack J. Dongarra, Sérgio Brissos, João Teixeira
Publisher	Springer
Pages	3-16
Number of pages	14
ISBN (Print)	9783030504328
DOIs	https://doi.org/10.1007/978-3-030-50433-5_1
Publication status	Published - 1 Jan 2020
MoE publication type	A4 Conference publication
Event	International Conference on Computational Science - Amsterdam, Netherlands Duration: 3 Jun 2020 → 5 Jun 2020 Conference number: 20

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12142 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	International Conference on Computational Science
Abbreviated title	ICCS
Country/Territory	Netherlands
City	Amsterdam
Period	03/06/2020 → 05/06/2020

Funding

The research leading to these results has received funding from the European Union?s Horizon2020 research and innovation programme under the EPEEC project, grant agreement No 801051. The work was also supported by the Academy of Finland (Flagship programme: Finnish Center for Artificial Intelligence, FCAI; grants 319264, 292334). We acknowledge PRACE for awarding us access to Hazel Hen at GCS@HLRS, Germany.

Access to Document

10.1007/978-3-030-50433-5_1

https://arxiv.org/abs/2004.02561

Cite this

Vander Aa, T., Qin, X., Blomstedt, P., Wuyts, R., Verachtert, W., & Kaski, S. (2020). A high-performance implementation of Bayesian matrix factorization with limited communication. In V. V. Krzhizhanovskaya, G. Závodszky, M. H. Lees, P. M. A. Sloot, P. M. A. Sloot, P. M. A. Sloot, J. J. Dongarra, S. Brissos, & J. Teixeira (Eds.), Computational Science – ICCS 2020 - 20th International Conference, Proceedings (pp. 3-16). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12142 LNCS). Springer. https://doi.org/10.1007/978-3-030-50433-5_1

Vander Aa, Tom ; Qin, Xiangju ; Blomstedt, Paul et al. / A high-performance implementation of Bayesian matrix factorization with limited communication. Computational Science – ICCS 2020 - 20th International Conference, Proceedings. editor / Valeria V. Krzhizhanovskaya ; Gábor Závodszky ; Michael H. Lees ; Peter M.A. Sloot ; Peter M.A. Sloot ; Peter M.A. Sloot ; Jack J. Dongarra ; Sérgio Brissos ; João Teixeira. Springer, 2020. pp. 3-16 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{c343f42dfed446a9898ca1d33a6df931,

title = "A high-performance implementation of Bayesian matrix factorization with limited communication",

abstract = "Matrix factorization is a very common machine learning technique in recommender systems. Bayesian Matrix Factorization (BMF) algorithms would be attractive because of their ability to quantify uncertainty in their predictions and avoid over-fitting, combined with high prediction accuracy. However, they have not been widely used on large-scale data because of their prohibitive computational cost. In recent work, efforts have been made to reduce the cost, both by improving the scalability of the BMF algorithm as well as its implementation, but so far mainly separately. In this paper we show that the state-of-the-art of both approaches to scalability can be combined. We combine the recent highly-scalable Posterior Propagation algorithm for BMF, which parallelizes computation of blocks of the matrix, with a distributed BMF implementation that users asynchronous communication within each block. We show that the combination of the two methods gives substantial improvements in the scalability of BMF on web-scale datasets, when the goal is to reduce the wall-clock time.",

author = "\{Vander Aa\}, Tom and Xiangju Qin and Paul Blomstedt and Roel Wuyts and Wilfried Verachtert and Samuel Kaski",

note = "| openaire: EC/H2020/801051/EU//EPEEC ; International Conference on Computational Science, ICCS ; Conference date: 03-06-2020 Through 05-06-2020",

year = "2020",

month = jan,

day = "1",

doi = "10.1007/978-3-030-50433-5\_1",

language = "English",

isbn = "9783030504328",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "3--16",

editor = "Krzhizhanovskaya, \{Valeria V.\} and G{\'a}bor Z{\'a}vodszky and Lees, \{Michael H.\} and Sloot, \{Peter M.A.\} and Sloot, \{Peter M.A.\} and Sloot, \{Peter M.A.\} and Dongarra, \{Jack J.\} and S{\'e}rgio Brissos and Jo{\~a}o Teixeira",

booktitle = "Computational Science – ICCS 2020 - 20th International Conference, Proceedings",

address = "Germany",

}

Vander Aa, T, Qin, X, Blomstedt, P, Wuyts, R, Verachtert, W & Kaski, S 2020, A high-performance implementation of Bayesian matrix factorization with limited communication. in VV Krzhizhanovskaya, G Závodszky, MH Lees, PMA Sloot, PMA Sloot, PMA Sloot, JJ Dongarra, S Brissos & J Teixeira (eds), Computational Science – ICCS 2020 - 20th International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12142 LNCS, Springer, pp. 3-16, International Conference on Computational Science, Amsterdam, Netherlands, 03/06/2020. https://doi.org/10.1007/978-3-030-50433-5_1

A high-performance implementation of Bayesian matrix factorization with limited communication. / Vander Aa, Tom; Qin, Xiangju; Blomstedt, Paul et al.
Computational Science – ICCS 2020 - 20th International Conference, Proceedings. ed. / Valeria V. Krzhizhanovskaya; Gábor Závodszky; Michael H. Lees; Peter M.A. Sloot; Peter M.A. Sloot; Peter M.A. Sloot; Jack J. Dongarra; Sérgio Brissos; João Teixeira. Springer, 2020. p. 3-16 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12142 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

TY - GEN

T1 - A high-performance implementation of Bayesian matrix factorization with limited communication

AU - Vander Aa, Tom

AU - Qin, Xiangju

AU - Blomstedt, Paul

AU - Wuyts, Roel

AU - Verachtert, Wilfried

AU - Kaski, Samuel

N1 - Conference code: 20

PY - 2020/1/1

Y1 - 2020/1/1

N2 - Matrix factorization is a very common machine learning technique in recommender systems. Bayesian Matrix Factorization (BMF) algorithms would be attractive because of their ability to quantify uncertainty in their predictions and avoid over-fitting, combined with high prediction accuracy. However, they have not been widely used on large-scale data because of their prohibitive computational cost. In recent work, efforts have been made to reduce the cost, both by improving the scalability of the BMF algorithm as well as its implementation, but so far mainly separately. In this paper we show that the state-of-the-art of both approaches to scalability can be combined. We combine the recent highly-scalable Posterior Propagation algorithm for BMF, which parallelizes computation of blocks of the matrix, with a distributed BMF implementation that users asynchronous communication within each block. We show that the combination of the two methods gives substantial improvements in the scalability of BMF on web-scale datasets, when the goal is to reduce the wall-clock time.

AB - Matrix factorization is a very common machine learning technique in recommender systems. Bayesian Matrix Factorization (BMF) algorithms would be attractive because of their ability to quantify uncertainty in their predictions and avoid over-fitting, combined with high prediction accuracy. However, they have not been widely used on large-scale data because of their prohibitive computational cost. In recent work, efforts have been made to reduce the cost, both by improving the scalability of the BMF algorithm as well as its implementation, but so far mainly separately. In this paper we show that the state-of-the-art of both approaches to scalability can be combined. We combine the recent highly-scalable Posterior Propagation algorithm for BMF, which parallelizes computation of blocks of the matrix, with a distributed BMF implementation that users asynchronous communication within each block. We show that the combination of the two methods gives substantial improvements in the scalability of BMF on web-scale datasets, when the goal is to reduce the wall-clock time.

UR - https://www.scopus.com/pages/publications/85087273462

U2 - 10.1007/978-3-030-50433-5_1

DO - 10.1007/978-3-030-50433-5_1

M3 - Conference article in proceedings

AN - SCOPUS:85087273462

SN - 9783030504328

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 3

EP - 16

BT - Computational Science – ICCS 2020 - 20th International Conference, Proceedings

A2 - Krzhizhanovskaya, Valeria V.

A2 - Závodszky, Gábor

A2 - Lees, Michael H.

A2 - Sloot, Peter M.A.

A2 - Dongarra, Jack J.

A2 - Brissos, Sérgio

A2 - Teixeira, João

PB - Springer

T2 - International Conference on Computational Science

Y2 - 3 June 2020 through 5 June 2020

ER -

Vander Aa T, Qin X, Blomstedt P, Wuyts R, Verachtert W, Kaski S. A high-performance implementation of Bayesian matrix factorization with limited communication. In Krzhizhanovskaya VV, Závodszky G, Lees MH, Sloot PMA, Sloot PMA, Sloot PMA, Dongarra JJ, Brissos S, Teixeira J, editors, Computational Science – ICCS 2020 - 20th International Conference, Proceedings. Springer. 2020. p. 3-16. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-50433-5_1

A high-performance implementation of Bayesian matrix factorization with limited communication

Abstract

Publication series

Conference

Funding

Access to Document

Other files and links

Fingerprint

Interactive machine learning from multiple biodata sources

Interactive machine learning from multiple biodata sources

Cite this

A high-performance implementation of Bayesian matrix factorization with limited communication

Abstract

Publication series

Conference

Funding

Access to Document

Other files and links

Fingerprint

Projects

Interactive machine learning from multiple biodata sources

Interactive machine learning from multiple biodata sources

Cite this