Image specificity

Mainak Jas, Devi Parikh

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

23 Citations (Scopus)

Abstract

For some images, descriptions written by multiple people are consistent with each other. But for other images, descriptions across people vary considerably. In other words, some images are specific - they elicit consistent descriptions from different people - while other images are ambiguous. Applications involving images and text can benefit from an understanding of which images are specific and which ones are ambiguous. For instance, consider text-based image retrieval. If a query description is moderately similar to the caption (or reference description) of an ambiguous image, that query may be considered a decent match to the image. But if the image is very specific, a moderate similarity between the query and the reference description may not be sufficient to retrieve the image. In this paper, we introduce the notion of image specificity. We present two mechanisms to measure specificity given multiple descriptions of an image: an automated measure and a measure that relies on human judgement. We analyze image specificity with respect to image content and properties to better understand what makes an image specific. We then train models to automatically predict the specificity of an image from image features alone without requiring textual descriptions of the image. Finally, we show that modeling image specificity leads to improvements in a text-based image retrieval application.

Original languageEnglish
Title of host publicationIEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015
Pages2727-2736
Number of pages10
Volume07-12-June-2015
ISBN (Electronic)9781467369640
DOIs
Publication statusPublished - 14 Oct 2015
MoE publication typeA4 Article in a conference publication
EventIEEE Conference on Computer Vision and Pattern Recognition - Boston, United States
Duration: 7 Jun 201512 Jun 2015

Conference

ConferenceIEEE Conference on Computer Vision and Pattern Recognition
Abbreviated titleCVPR
CountryUnited States
CityBoston
Period07/06/201512/06/2015

Fingerprint Dive into the research topics of 'Image specificity'. Together they form a unique fingerprint.

Cite this