CascadeXML: Rethinking Transformers for End-to-end Multi-resolution Training in Extreme Multi-label Classification

Siddhant Kharbanda, Atmadeep Banerjee, Erik Schultheis, Rohit Babbar

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

8 Citations (Scopus)

Abstract

Extreme Multi-label Text Classification (XMC) involves learning a classifier that can assign an input with a subset of most relevant labels from millions of label choices. Recent approaches, such as XR-Transformer and LightXML, leverage a transformer instance to achieve state-of-the-art performance. However, in this process, these approaches need to make various trade-offs between performance and computational requirements. A major shortcoming, as compared to the Bi-LSTM based AttentionXML, is that they fail to keep separate feature representations for each resolution in a label tree. We thus propose CascadeXML, an end-to-end multi-resolution learning pipeline, which can harness the multi-layered architecture of a transformer model for attending to different label resolutions with separate feature representations. CascadeXML significantly outperforms all existing approaches with non-trivial gains obtained on benchmark datasets consisting of up to three million labels.
Original languageEnglish
Title of host publicationAdvances in Neural Information Processing Systems 35 (NeurIPS 2022)
EditorsS. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh
PublisherMorgan Kaufmann Publishers
Number of pages14
ISBN (Print)978-1-7138-7108-8
Publication statusPublished - 2022
MoE publication typeA4 Conference publication
EventConference on Neural Information Processing Systems - New Orleans, United States
Duration: 28 Nov 20229 Dec 2022
Conference number: 36
https://nips.cc/

Publication series

NameAdvances in Neural Information Processing Systems
PublisherMorgan Kaufmann Publishers
Volume35
ISSN (Print)1049-5258

Conference

ConferenceConference on Neural Information Processing Systems
Abbreviated titleNeurIPS
Country/TerritoryUnited States
CityNew Orleans
Period28/11/202209/12/2022
Internet address

Fingerprint

Dive into the research topics of 'CascadeXML: Rethinking Transformers for End-to-end Multi-resolution Training in Extreme Multi-label Classification'. Together they form a unique fingerprint.

Cite this