Projects per year
Multimodal recognition has recently become more attractive and common method in multimedia information retrieval. In many cases it shows better recognition results than using only unimodal methods. Most of current multimodal recognition methods still depend on unimodal recognition results. Therefore, in order to get better recognition performance, it is important to choose suitable features and classification models for each unimodal recognition task. In this paper, we research several unimodal recognition methods, features for them and their combination techniques, in the application setup of concept detection in image-text data. For image features, we use GoogLeNet deep convolutional neural network (DCNN) activation features and semantic concept vectors. For text features, we use simple binary vectors for tags and word2vec vectors. As the concept detection model, we apply the Multimodal Deep Boltzmann Machine (DBM) model and the Support Vector Machine (SVM) with the linear homogeneous kernel map and the non-linear radial basis function (RBF) kernel. The experimental results with the MIRFLICKR-1M data set show that the Multimodal DBM or the non-linear SVM approaches produce equally good results within the margins of statitistical variation.
|Title of host publication||2016 14th International Workshop on Content-Based Multimedia Indexing, CBMI 2016|
|Publication status||Published - 27 Jun 2016|
|MoE publication type||A4 Article in a conference publication|
|Event||International Workshop on Content-Based Multimedia Indexing - Bucharest, Romania|
Duration: 15 Jun 2016 → 17 Jun 2016
Conference number: 14
|Workshop||International Workshop on Content-Based Multimedia Indexing|
|Period||15/06/2016 → 17/06/2016|
FingerprintDive into the research topics of 'Comparing and combining unimodal methods for multimodal recognition'. Together they form a unique fingerprint.
- 1 Finished
Finnish centre of excellence in computational inference research
Xu, Y., Rintanen, J., Kaski, S., Anwer, R., Parviainen, P., Soare, M., Vuollekoski, H., Rezazadegan Tavakoli, H., Peltola, T., Blomstedt, P., Puranen, S., Dutta, R., Gebser, M., Mononen, T., Bogaerts, B., Tasharrofi, S., Weinzierl, A., Yang, Z. & Pesonen, H.
01/01/2015 → 31/12/2017
Project: Academy of Finland: Other research funding