Comparing and combining unimodal methods for multimodal recognition

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference contributionScientificvertaisarvioitu


Multimodal recognition has recently become more attractive and common method in multimedia information retrieval. In many cases it shows better recognition results than using only unimodal methods. Most of current multimodal recognition methods still depend on unimodal recognition results. Therefore, in order to get better recognition performance, it is important to choose suitable features and classification models for each unimodal recognition task. In this paper, we research several unimodal recognition methods, features for them and their combination techniques, in the application setup of concept detection in image-text data. For image features, we use GoogLeNet deep convolutional neural network (DCNN) activation features and semantic concept vectors. For text features, we use simple binary vectors for tags and word2vec vectors. As the concept detection model, we apply the Multimodal Deep Boltzmann Machine (DBM) model and the Support Vector Machine (SVM) with the linear homogeneous kernel map and the non-linear radial basis function (RBF) kernel. The experimental results with the MIRFLICKR-1M data set show that the Multimodal DBM or the non-linear SVM approaches produce equally good results within the margins of statitistical variation.

Otsikko2016 14th International Workshop on Content-Based Multimedia Indexing, CBMI 2016
KustantajaIEEE Computer Society
ISBN (elektroninen)9781467386951
DOI - pysyväislinkit
TilaJulkaistu - 27 kesäkuuta 2016
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisuussa
TapahtumaInternational Workshop on Content-Based Multimedia Indexing - Bucharest, Romania
Kesto: 15 kesäkuuta 201617 kesäkuuta 2016
Konferenssinumero: 14


WorkshopInternational Workshop on Content-Based Multimedia Indexing

Sormenjälki Sukella tutkimusaiheisiin 'Comparing and combining unimodal methods for multimodal recognition'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä