A strong and robust baseline for text-image matching

Fangyu Liu, Rongtian Ye

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference contributionScientificvertaisarvioitu

3 Sitaatiot (Scopus)


We review the current schemes of text-image matching models and propose improvements for both training and inference. First, we empirically show limitations of two popular loss (sum and max-margin loss) widely used in training text-image embeddings and propose a trade-off: a kNN-margin loss which 1) utilizes information from hard negatives and 2) is robust to noise as all K-most hardest samples are taken into account, tolerating pseudo negatives and outliers. Second, we advocate the use of Inverted Softmax (IS) and Crossmodal Local Scaling (CSLS) during inference to mitigate the so-called hubness problem in high-dimensional embedding space, enhancing scores of all metrics by a large margin.

OtsikkoACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Student Research Workshop
KustantajaAssociation for Computational Linguistics (ACL)
ISBN (elektroninen)9781950737475
TilaJulkaistu - 2019
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisuussa
TapahtumaAnnual Meeting of the Association for Computational Linguistics: Student Research Workshop - Florence, Italia
Kesto: 28 heinäk. 20192 elok. 2019


WorkshopAnnual Meeting of the Association for Computational Linguistics


Sukella tutkimusaiheisiin 'A strong and robust baseline for text-image matching'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä