Interpolation consistency training for semi-supervised learning

Vikas Verma; Alex Lamb; Juho Kannala; Yoshua Bengio; David Lopez-Paz

doi:10.24963/ijcai.2019/504

Interpolation consistency training for semi-supervised learning

Vikas Verma, Alex Lamb, Juho Kannala, Yoshua Bengio, David Lopez-Paz

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

Abstract

We introduce Interpolation Consistency Training (ICT), a simple and computation efficient algorithm for training Deep Neural Networks in the semi-supervised learning paradigm. ICT encourages the prediction at an interpolation of unlabeled points to be consistent with the interpolation of the predictions at those points. In classification problems, ICT moves the decision boundary to low-density regions of the data distribution. Our experiments show that ICT achieves state-of-the-art performance when applied to standard neural network architectures on the CIFAR-10 and SVHN benchmark datasets.

Original language	English
Title of host publication	Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)
Publisher	IJCAI
Pages	3635-3641
ISBN (Electronic)	978-0-9992411-4-1
DOIs	https://doi.org/10.24963/ijcai.2019/504
Publication status	Published - 2019
MoE publication type	A4 Conference publication
Event	International Joint Conference on Artificial Intelligence - Venetian Macao Resort Hotel, Macao, China Duration: 10 Aug 2019 → 16 Aug 2019 Conference number: 28 https://ijcai19.org/ http://ijcai19.org/

Conference

Conference	International Joint Conference on Artificial Intelligence
Abbreviated title	IJCAI
Country/Territory	China
City	Macao
Period	10/08/2019 → 16/08/2019
Internet address	https://ijcai19.org/ http://ijcai19.org/

Keywords

Deep Learning

Access to Document

10.24963/ijcai.2019/504

https://arxiv.org/abs/1903.03825

Cite this

@inproceedings{ad3e4cf616bd42e8b4edd2b0918b3dda,

title = "Interpolation consistency training for semi-supervised learning",

abstract = "We introduce Interpolation Consistency Training (ICT), a simple and computation efficient algorithm for training Deep Neural Networks in the semi-supervised learning paradigm. ICT encourages the prediction at an interpolation of unlabeled points to be consistent with the interpolation of the predictions at those points. In classification problems, ICT moves the decision boundary to low-density regions of the data distribution. Our experiments show that ICT achieves state-of-the-art performance when applied to standard neural network architectures on the CIFAR-10 and SVHN benchmark datasets.",

keywords = "Deep Learning",

author = "Vikas Verma and Alex Lamb and Juho Kannala and Yoshua Bengio and David Lopez-Paz",

year = "2019",

doi = "10.24963/ijcai.2019/504",

language = "English",

pages = "3635--3641",

booktitle = "Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)",

publisher = "IJCAI",

address = "United States",

note = "International Joint Conference on Artificial Intelligence, IJCAI ; Conference date: 10-08-2019 Through 16-08-2019",

url = "https://ijcai19.org/, http://ijcai19.org/",

}

Verma, V, Lamb, A, Kannala, J, Bengio, Y & Lopez-Paz, D 2019, Interpolation consistency training for semi-supervised learning. in Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19). IJCAI, pp. 3635-3641, International Joint Conference on Artificial Intelligence, Macao, China, 10/08/2019. https://doi.org/10.24963/ijcai.2019/504

Interpolation consistency training for semi-supervised learning. / Verma, Vikas; Lamb, Alex; Kannala, Juho et al.
Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19). IJCAI, 2019. p. 3635-3641.

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

TY - GEN

T1 - Interpolation consistency training for semi-supervised learning

AU - Verma, Vikas

AU - Lamb, Alex

AU - Kannala, Juho

AU - Bengio, Yoshua

AU - Lopez-Paz, David

N1 - Conference code: 28

PY - 2019

Y1 - 2019

N2 - We introduce Interpolation Consistency Training (ICT), a simple and computation efficient algorithm for training Deep Neural Networks in the semi-supervised learning paradigm. ICT encourages the prediction at an interpolation of unlabeled points to be consistent with the interpolation of the predictions at those points. In classification problems, ICT moves the decision boundary to low-density regions of the data distribution. Our experiments show that ICT achieves state-of-the-art performance when applied to standard neural network architectures on the CIFAR-10 and SVHN benchmark datasets.

AB - We introduce Interpolation Consistency Training (ICT), a simple and computation efficient algorithm for training Deep Neural Networks in the semi-supervised learning paradigm. ICT encourages the prediction at an interpolation of unlabeled points to be consistent with the interpolation of the predictions at those points. In classification problems, ICT moves the decision boundary to low-density regions of the data distribution. Our experiments show that ICT achieves state-of-the-art performance when applied to standard neural network architectures on the CIFAR-10 and SVHN benchmark datasets.

KW - Deep Learning

UR - https://www.ijcai.org/proceedings/2019/

U2 - 10.24963/ijcai.2019/504

DO - 10.24963/ijcai.2019/504

M3 - Conference article in proceedings

SP - 3635

EP - 3641

BT - Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)

PB - IJCAI

T2 - International Joint Conference on Artificial Intelligence

Y2 - 10 August 2019 through 16 August 2019

ER -

Interpolation consistency training for semi-supervised learning

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Algorithms for Data-Efficient Training of Deep Neural Networks

Cite this

Interpolation consistency training for semi-supervised learning

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Press/Media

Algorithms for Data-Efficient Training of Deep Neural Networks

Cite this