Projects per year
Abstract
E(n)-Equivariant Graph Neural Networks (EGNNs) are among the most widely used and successful models for representation learning on geometric graphs (e.g., 3D molecules). However, while the expressivity of EGNNs has been explored in terms of geometric variants of the Weisfeiler-Leman isomorphism test, characterizing their generalization capability remains open. In this work, we establish the first generalization bound for EGNNs. Our bound depicts a dependence on the weighted sum of logarithms of the spectral norms of the weight matrices (EGNN parameters). In addition, our main result reveals interesting novel insights: i) the spectral norms of the initial layers may impact generalization more than the final ones; ii) ε-normalization is beneficial to generalization ' confirming prior empirical evidence. We leverage these insights to introduce a spectral norm regularizer tailored to EGNNs. Experiments on real-world datasets substantiate our analysis, demonstrating a high correlation between theoretical and empirical generalization gaps and the effectiveness of the proposed regularization scheme.
Original language | English |
---|---|
Pages (from-to) | 23159-23186 |
Number of pages | 28 |
Journal | Proceedings of Machine Learning Research |
Volume | 235 |
Publication status | Published - 2024 |
MoE publication type | A4 Conference publication |
Event | International Conference on Machine Learning - Vienna, Austria Duration: 21 Jul 2024 → 27 Jul 2024 Conference number: 41 |
Fingerprint
Dive into the research topics of 'On the Generalization of Equivariant Graph Neural Networks'. Together they form a unique fingerprint.Projects
- 1 Finished
-
-: Finnish Center for Artificial Intelligence
Kaski, S. (Principal investigator)
01/01/2019 → 31/12/2022
Project: Academy of Finland: Other research funding