Towards Domain-Agnostic Contrastive Learning

Vikas Verma, Minh-Thang Luong, Kenji Kawaguchi, Hieu Pham, Quoc V. Le

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference article in proceedingsScientificvertaisarvioitu

29 Lataukset (Pure)

Abstrakti

Despite recent successes, most contrastive self-supervised learning methods are domain-specific, relying heavily on data augmentation techniques that require knowledge about a particular domain, such as image cropping and rotation. To overcome such limitation, we propose a domain-agnostic approach to contrastive learning, named DACL, that is applicable to problems where domain-specific data augmentations are not readily available. Key to our approach is the use of Mixup noise to create similar and dissimilar examples by mixing data samples differently either at the input or hidden-state levels. We theoretically analyze our method and show advantages over the Gaussian-noise based contrastive learning approach. To demonstrate the effectiveness of DACL, we conduct experiments across various domains such as tabular data, images, and graphs. Our results show that DACL not only outperforms other domain-agnostic noising methods, such as Gaussian-noise, but also combines well with domain-specific methods, such as SimCLR, to improve self-supervised visual representation learning.
AlkuperäiskieliEnglanti
OtsikkoProceedings of the 38 th International Conference on Machine Learning
KustantajaJMLR
Sivumäärä12
TilaJulkaistu - 2021
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisussa
TapahtumaInternational Conference on Machine Learning - Virtual, Online
Kesto: 18 heinäk. 202124 heinäk. 2021
Konferenssinumero: 38

Julkaisusarja

NimiProceedings of Machine Learning Research
KustantajaPMLR
Vuosikerta139
ISSN (elektroninen)2640-3498

Conference

ConferenceInternational Conference on Machine Learning
LyhennettäICML
KaupunkiVirtual, Online
Ajanjakso18/07/202124/07/2021

Sormenjälki

Sukella tutkimusaiheisiin 'Towards Domain-Agnostic Contrastive Learning'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä