Multi-node Training for StyleGAN2

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference article in proceedingsScientificvertaisarvioitu

205 Lataukset (Pure)

Abstrakti

StyleGAN2 is a Tensorflow-based Generative Adversarial Network (GAN) framework that represents the state-of-the-art in generative image modelling. The current release of StyleGAN2 implements multi-GPU training via Tensorflow’s device contexts which limits data parallelism to a single node. In this work, a data-parallel multi-node training capability is implemented in StyleGAN2 via Horovod which enables harnessing the compute capability of larger cluster architectures. We demonstrate that the new Horovod-based communication outperforms the previous context approach on a single node. Furthermore, we demonstrate that the multi-node training does not compromise the accuracy of StyleGAN2 for a constant effective batch size. Finally, we report strong and weak scaling of the new implementation up to 64 NVIDIA Tesla A100 GPUs distributed across eight NVIDIA DGX A100 nodes, demonstrating the utility of the approach at scale.

AlkuperäiskieliEnglanti
OtsikkoPattern Recognition. ICPR International Workshops and Challenges, 2021, Proceedings
ToimittajatAlberto Del Bimbo, Rita Cucchiara, Stan Sclaroff, Giovanni Maria Farinella, Tao Mei, Marco Bertini, Hugo Jair Escalante, Roberto Vezzani
KustantajaSpringer
Sivut677-684
Sivumäärä8
ISBN (painettu)978-3-030-68762-5
DOI - pysyväislinkit
TilaJulkaistu - 2021
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisussa
TapahtumaInternational Conference on Pattern Recognition - Virtual, Online, Milan, Italia
Kesto: 10 tammik. 202115 tammik. 2021
Konferenssinumero: 25

Julkaisusarja

NimiLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
KustantajaSpringer
Vuosikerta12661 LNCS
ISSN (painettu)0302-9743
ISSN (elektroninen)1611-3349

Conference

ConferenceInternational Conference on Pattern Recognition
LyhennettäICPR
Maa/AlueItalia
KaupunkiMilan
Ajanjakso10/01/202115/01/2021

Sormenjälki

Sukella tutkimusaiheisiin 'Multi-node Training for StyleGAN2'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä