InceptionXML: A Lightweight Framework with Synchronized Negative Sampling for Short Text Extreme Classification

Siddhant Kharbanda, Atmadeep Banerjee, Devaansh Gupta, Akash Palrecha, Rohit Babbar

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

1 Citation (Scopus)

Abstract

Automatic annotation of short-text data to a large number of target labels, referred to as Short Text Extreme Classification, has found numerous applications including prediction of related searches and product recommendation. In this paper, we propose a convolutional architecture InceptionXML which is light-weight, yet powerful, and robust to the inherent lack of word-order in short-text queries encountered in search and recommendation. We demonstrate the efficacy of applying convolutions by recasting the operation along the embedding dimension instead of the word dimension as applied in conventional CNNs for text classification. Towards scaling our model to datasets with millions of labels, we also propose SyncXML pipeline which improves upon the shortcomings of the recently proposed dynamic hard-negative mining technique for label shortlisting by synchronizing the label-shortlister and extreme classifier. SyncXML not only reduces the inference time to half but is also an order of magnitude smaller than state-of-the-art Astec in terms of model size. Through a comprehensive empirical comparison, we show that not only can InceptionXML outperform existing approaches on benchmark datasets but also the transformer baselines requiring only 2% FLOPs. The code for InceptionXML is available at https://github.com/xmc-aalto.
Original languageEnglish
Title of host publicationSIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval
PublisherACM
Pages760–769
ISBN (Electronic)978-1-4503-9408-6
DOIs
Publication statusPublished - 19 Jul 2023
MoE publication typeA4 Conference publication
EventInternational ACM SIGIR Conference on Research and Development in Information Retrieval - Taipei, Taiwan, Republic of China
Duration: 23 Jul 202327 Jul 2023
Conference number: 46

Conference

ConferenceInternational ACM SIGIR Conference on Research and Development in Information Retrieval
Abbreviated titleSIGIR
Country/TerritoryTaiwan, Republic of China
CityTaipei
Period23/07/202327/07/2023

Fingerprint

Dive into the research topics of 'InceptionXML: A Lightweight Framework with Synchronized Negative Sampling for Short Text Extreme Classification'. Together they form a unique fingerprint.

Cite this