Projekteja vuodessa
Abstrakti
Sequential learning paradigms pose challenges for gradient-based deep learning due to difficulties incorporating new data and retaining prior knowledge. While Gaussian processes elegantly tackle these problems, they struggle with scalability and handling rich inputs, such as images. To address these issues, we introduce a technique that converts neural networks from weight space to function space, through a dual parameterization. Our parameterization offers: (i) a way to scale function-space methods to large data sets via sparsification, (ii) retention of prior knowledge when access to past data is limited, and (iii) a mechanism to incorporate new data without retraining. Our experiments demonstrate that we can retain knowledge in continual learning and incorporate new data efficiently. We further show its strengths in uncertainty quantification and guiding exploration in model-based RL. Further information and code is available on the project website.
Alkuperäiskieli | Englanti |
---|---|
Sivumäärä | 29 |
Tila | Julkaistu - 2024 |
OKM-julkaisutyyppi | Ei sovellu |
Tapahtuma | International Conference on Learning Representations - Messe Wien Exhibition and Congress Center, Vienna, Itävalta Kesto: 7 toukok. 2024 → 11 toukok. 2024 Konferenssinumero: 12 https://iclr.cc/ |
Conference
Conference | International Conference on Learning Representations |
---|---|
Lyhennettä | ICLR |
Maa/Alue | Itävalta |
Kaupunki | Vienna |
Ajanjakso | 07/05/2024 → 11/05/2024 |
www-osoite |
Sormenjälki
Sukella tutkimusaiheisiin 'FUNCTION-SPACE PARAMETERIZATION OF NEURAL NETWORKS FOR SEQUENTIAL LEARNING'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.-
Solin Arno /AoF Fellow Salary: Probabilistic principles for latent space exploration in deep learning
Solin, A. (Vastuullinen tutkija) & Mereu, R. (Projektin jäsen)
01/09/2021 → 31/08/2026
Projekti: Academy of Finland: Other research funding
-
-: Finnish Center for Artificial Intelligence
Kaski, S. (Vastuullinen tutkija)
01/01/2019 → 31/12/2022
Projekti: Academy of Finland: Other research funding