SketchTransfer: A challenging new task for exploring detail-invariance and the abstractions learned by deep networks

Alex Lamb, Sherjil Ozair, Vikas Verma, David Ha

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference article in proceedingsScientificvertaisarvioitu

6 Sitaatiot (Scopus)


Deep networks have achieved excellent results in perceptual tasks, yet their ability to generalize to variations not seen during training has come under increasing scrutiny. In this work we focus on their ability to have invariance towards the presence or absence of details. For example, humans are able to watch cartoons, which are missing many visual details, without being explicitly trained to do so. As another example, 3D rendering software is a relatively recent development, yet people are able to understand such rendered scenes even though they are missing details (consider a film like Toy Story). The failure of ma- chine learning algorithms to do this indicates a significant gap in generalization between human abilities and the abilities of deep networks. We propose a dataset that will make it easier to study the detail-invariance problem concretely. We produce a concrete task for this: SketchTransfer, and we show that state-of-the-art domain transfer algorithms still struggle with this task. The state-of-the-art technique which achieves over 95% on MNIST → SVHN transfer only achieves 59% accuracy on the SketchTransfer task, which is much better than random (11% accuracy) but falls short of the 87% accuracy of a classifier trained directly on labeled sketches. This indicates that this task is approachable with today's best methods but has substantial room for improvement.

OtsikkoProceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020
ISBN (elektroninen)9781728165530
DOI - pysyväislinkit
TilaJulkaistu - maalisk. 2020
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisussa
TapahtumaIEEE Winter Conference on Applications of Computer Vision - Snowmass Village, Yhdysvallat
Kesto: 1 maalisk. 20205 maalisk. 2020


ConferenceIEEE Winter Conference on Applications of Computer Vision
KaupunkiSnowmass Village


Sukella tutkimusaiheisiin 'SketchTransfer: A challenging new task for exploring detail-invariance and the abstractions learned by deep networks'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä