CascadeXML: Rethinking Transformers for End-to-end Multi-resolution Training in Extreme Multi-label Classification

Siddhant Kharbanda, Atmadeep Banerjee, Erik Schultheis, Rohit Babbar

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference article in proceedingsScientificvertaisarvioitu

6 Sitaatiot (Scopus)

Abstrakti

Extreme Multi-label Text Classification (XMC) involves learning a classifier that can assign an input with a subset of most relevant labels from millions of label choices. Recent approaches, such as XR-Transformer and LightXML, leverage a transformer instance to achieve state-of-the-art performance. However, in this process, these approaches need to make various trade-offs between performance and computational requirements. A major shortcoming, as compared to the Bi-LSTM based AttentionXML, is that they fail to keep separate feature representations for each resolution in a label tree. We thus propose CascadeXML, an end-to-end multi-resolution learning pipeline, which can harness the multi-layered architecture of a transformer model for attending to different label resolutions with separate feature representations. CascadeXML significantly outperforms all existing approaches with non-trivial gains obtained on benchmark datasets consisting of up to three million labels.
AlkuperäiskieliEnglanti
OtsikkoAdvances in Neural Information Processing Systems 35 (NeurIPS 2022)
ToimittajatS. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh
KustantajaMorgan Kaufmann Publishers
Sivumäärä14
ISBN (painettu)978-1-7138-7108-8
TilaJulkaistu - 2022
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisussa
TapahtumaConference on Neural Information Processing Systems - New Orleans, Yhdysvallat
Kesto: 28 marrask. 20229 jouluk. 2022
Konferenssinumero: 36
https://nips.cc/

Julkaisusarja

NimiAdvances in Neural Information Processing Systems
KustantajaMorgan Kaufmann Publishers
Vuosikerta35
ISSN (painettu)1049-5258

Conference

ConferenceConference on Neural Information Processing Systems
LyhennettäNeurIPS
Maa/AlueYhdysvallat
KaupunkiNew Orleans
Ajanjakso28/11/202209/12/2022
www-osoite

Sormenjälki

Sukella tutkimusaiheisiin 'CascadeXML: Rethinking Transformers for End-to-end Multi-resolution Training in Extreme Multi-label Classification'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä