Memory-Based Dual Gaussian Processes for Sequential Learning

Paul E. Chang; Prakhar Verma; S.T. John; Arno Solin; Mohammad Emtiyaz Khan

Memory-Based Dual Gaussian Processes for Sequential Learning

Paul E. Chang, Prakhar Verma, S.T. John, Arno Solin, Mohammad Emtiyaz Khan

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussa › Conference article in proceedings › Scientific › vertaisarvioitu

56 Lataukset (Pure)

Abstrakti

Sequential learning with Gaussian processes (GPs) is challenging when access to past data is limited, for example, in continual and active learning. In such cases, errors can accumulate over time due to inaccuracies in the posterior, hyperparameters, and inducing points, making accurate learning challenging. Here, we present a method to keep all such errors in check using the recently proposed dual sparse variational GP. Our method enables accurate inference for generic likelihoods and improves learning by actively building and updating a memory of past data. We demonstrate its effectiveness in several applications involving Bayesian optimization, active learning, and continual learning.

Alkuperäiskieli	Englanti
Otsikko	Proceedings of the 40th International Conference on Machine Learning
Toimittajat	Andread Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, Jonathan Scarlett
Kustantaja	JMLR
Sivut	4035-4054
Sivumäärä	20
Tila	Julkaistu - heinäk. 2023
OKM-julkaisutyyppi	A4 Artikkeli konferenssijulkaisussa
Tapahtuma	International Conference on Machine Learning - Honolulu, Yhdysvallat Kesto: 23 heinäk. 2023 → 29 heinäk. 2023 Konferenssinumero: 40

Julkaisusarja

Nimi	Proceedings of Machine Learning Research
Kustantaja	PMLR
Vuosikerta	202
ISSN (elektroninen)	2640-3498

Conference

Conference	International Conference on Machine Learning
Lyhennettä	ICML
Maa/Alue	Yhdysvallat
Kaupunki	Honolulu
Ajanjakso	23/07/2023 → 29/07/2023

Pääsy asiakirjaan

SCI_Chang_etal_ICML_2023Final published version, 5,5 MBLisenssi: CC BY

https://proceedings.mlr.press/v202/chang23a.htmlLisenssi: CC BY

1 Aktiivinen
2 Päättynyt

Solin Arno /AoF Fellow Salary: Probabilistic principles for latent space exploration in deep learning
Solin, A. (Vastuullinen tutkija) & Mereu, R. (Projektin jäsen)
01/09/2021 → 31/08/2026
Projekti: Academy of Finland: Other research funding
-: Matalat mallit syvien apuna konenäössä
Solin, A. (Vastuullinen tutkija), Wang, H. (Projektin jäsen), Tamir, E. (Projektin jäsen), Trapp, M. (Projektin jäsen), Verma, P. (Projektin jäsen), Chang, P. (Projektin jäsen), Li, R. (Projektin jäsen) & Mereu, R. (Projektin jäsen)
01/09/2019 → 31/08/2023
Projekti: Academy of Finland: Other research funding
-: Finnish Center for Artificial Intelligence
Kaski, S. (Vastuullinen tutkija)
01/01/2019 → 31/12/2022
Projekti: Academy of Finland: Other research funding

Science-IT
Hakala, M. (Manager)
Perustieteiden korkeakoulu
Laitteistot/tilat: Facility

Siteeraa tätä

Chang, P. E., Verma, P., John, S. T., Solin, A., & Emtiyaz Khan, M. (2023). Memory-Based Dual Gaussian Processes for Sequential Learning. teoksessa A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, & J. Scarlett (Toimittajat), Proceedings of the 40th International Conference on Machine Learning (Sivut 4035-4054). (Proceedings of Machine Learning Research; Vuosikerta 202). JMLR. https://proceedings.mlr.press/v202/chang23a.html

Chang, Paul E. ; Verma, Prakhar ; John, S.T. et al. / Memory-Based Dual Gaussian Processes for Sequential Learning. Proceedings of the 40th International Conference on Machine Learning. Toimittaja / Andread Krause ; Emma Brunskill ; Kyunghyun Cho ; Barbara Engelhardt ; Sivan Sabato ; Jonathan Scarlett. JMLR, 2023. Sivut 4035-4054 (Proceedings of Machine Learning Research).

@inproceedings{c1741c38c5974f978341cd81f6017ab5,

title = "Memory-Based Dual Gaussian Processes for Sequential Learning",

abstract = "Sequential learning with Gaussian processes (GPs) is challenging when access to past data is limited, for example, in continual and active learning. In such cases, errors can accumulate over time due to inaccuracies in the posterior, hyperparameters, and inducing points, making accurate learning challenging. Here, we present a method to keep all such errors in check using the recently proposed dual sparse variational GP. Our method enables accurate inference for generic likelihoods and improves learning by actively building and updating a memory of past data. We demonstrate its effectiveness in several applications involving Bayesian optimization, active learning, and continual learning. ",

author = "Chang, {Paul E.} and Prakhar Verma and S.T. John and Arno Solin and {Emtiyaz Khan}, Mohammad",

year = "2023",

month = jul,

language = "English",

series = "Proceedings of Machine Learning Research",

publisher = "JMLR",

pages = "4035--4054",

editor = "Andread Krause and Emma Brunskill and Kyunghyun Cho and Barbara Engelhardt and Sivan Sabato and Jonathan Scarlett",

booktitle = "Proceedings of the 40th International Conference on Machine Learning",

address = "United States",

note = "International Conference on Machine Learning, ICML ; Conference date: 23-07-2023 Through 29-07-2023",

}

Chang, PE , Verma, P , John, ST , Solin, A & Emtiyaz Khan, M 2023, Memory-Based Dual Gaussian Processes for Sequential Learning. julkaisussa A Krause, E Brunskill, K Cho, B Engelhardt, S Sabato & J Scarlett (toim), Proceedings of the 40th International Conference on Machine Learning. Proceedings of Machine Learning Research, Vuosikerta 202, JMLR, Sivut 4035-4054, International Conference on Machine Learning, Honolulu, Hawaii, Yhdysvallat, 23/07/2023. <https://proceedings.mlr.press/v202/chang23a.html>

Memory-Based Dual Gaussian Processes for Sequential Learning. / Chang, Paul E.; Verma, Prakhar ; John, S.T. et al.
Proceedings of the 40th International Conference on Machine Learning. toim. / Andread Krause; Emma Brunskill; Kyunghyun Cho; Barbara Engelhardt; Sivan Sabato; Jonathan Scarlett. JMLR, 2023. s. 4035-4054 (Proceedings of Machine Learning Research; Vuosikerta 202).

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussa › Conference article in proceedings › Scientific › vertaisarvioitu

TY - GEN

T1 - Memory-Based Dual Gaussian Processes for Sequential Learning

AU - Chang, Paul E.

AU - Verma, Prakhar

AU - John, S.T.

AU - Solin, Arno

AU - Emtiyaz Khan, Mohammad

N1 - Conference code: 40

PY - 2023/7

Y1 - 2023/7

N2 - Sequential learning with Gaussian processes (GPs) is challenging when access to past data is limited, for example, in continual and active learning. In such cases, errors can accumulate over time due to inaccuracies in the posterior, hyperparameters, and inducing points, making accurate learning challenging. Here, we present a method to keep all such errors in check using the recently proposed dual sparse variational GP. Our method enables accurate inference for generic likelihoods and improves learning by actively building and updating a memory of past data. We demonstrate its effectiveness in several applications involving Bayesian optimization, active learning, and continual learning.

AB - Sequential learning with Gaussian processes (GPs) is challenging when access to past data is limited, for example, in continual and active learning. In such cases, errors can accumulate over time due to inaccuracies in the posterior, hyperparameters, and inducing points, making accurate learning challenging. Here, we present a method to keep all such errors in check using the recently proposed dual sparse variational GP. Our method enables accurate inference for generic likelihoods and improves learning by actively building and updating a memory of past data. We demonstrate its effectiveness in several applications involving Bayesian optimization, active learning, and continual learning.

M3 - Conference article in proceedings

T3 - Proceedings of Machine Learning Research

SP - 4035

EP - 4054

BT - Proceedings of the 40th International Conference on Machine Learning

A2 - Krause, Andread

A2 - Brunskill, Emma

A2 - Cho, Kyunghyun

A2 - Engelhardt, Barbara

A2 - Sabato, Sivan

A2 - Scarlett, Jonathan

PB - JMLR

T2 - International Conference on Machine Learning

Y2 - 23 July 2023 through 29 July 2023

ER -

Memory-Based Dual Gaussian Processes for Sequential Learning

Abstrakti

Julkaisusarja

Conference

Pääsy asiakirjaan

Sormenjälki

Projektit

Solin Arno /AoF Fellow Salary: Probabilistic principles for latent space exploration in deep learning

-: Matalat mallit syvien apuna konenäössä

-: Finnish Center for Artificial Intelligence

Laitteet

Science-IT

Siteeraa tätä