Image pseudo tag generation with Deep Boltzmann machine anc topic-concept similarity map

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference contributionScientificvertaisarvioitu


General purpose search engines are used for searching not only plain text but also multimedia information. In multimodal search, it is common to use multiple queries to find the demanded information in the different media modalities. In most cases, however, it is hard to prepare such multimodal search queries. In addition, the semantic connection between the individual modalities is often weak or totally lacking in such multimodal search. Hence, single modality searching makes it hard to find the searched for information in the multimodal domain. In this paper we improve the Deep Boltzmann Machine applied to multimodal search by using GoogLeNet deep convolutional neural network and semantic concept features. We also propose a supervised method to produce a similarity map between hidden topics in text documents and the visual concepts in corresponding images, and an unsupervised method which uses the hidden topics in the documents as pseudo labels. The model can be used to infer and generate pseudo tags for untagged input query images in order to complement an image-only query to a multimodal one. The classification results with pseudo tag inputs show in our experiments improvement compared to the original tag inputs.

Otsikko2017 International Joint Conference on Neural Networks, IJCNN 2017 - Proceedings
ISBN (elektroninen)9781509061815
DOI - pysyväislinkit
TilaJulkaistu - 30 kesäkuuta 2017
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisuussa
TapahtumaInternational Joint Conference on Neural Networks - Anchorage, Yhdysvallat
Kesto: 14 toukokuuta 201719 toukokuuta 2017


ConferenceInternational Joint Conference on Neural Networks

Sormenjälki Sukella tutkimusaiheisiin 'Image pseudo tag generation with Deep Boltzmann machine anc topic-concept similarity map'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä