EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications

Muhammad Maaz*, Abdelrahman Shaker, Hisham Cholakkal, Salman Khan, Syed Waqas Zamir, Rao Muhammad Anwer, Fahad Shahbaz Khan

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

23 Citations (Scopus)

Abstract

In the pursuit of achieving ever-increasing accuracy, large and complex neural networks are usually developed. Such models demand high computational resources and therefore cannot be deployed on edge devices. It is of great interest to build resource-efficient general purpose networks due to their usefulness in several application areas. In this work, we strive to effectively combine the strengths of both CNN and Transformer models and propose a new efficient hybrid architecture EdgeNeXt. Specifically in EdgeNeXt, we introduce split depth-wise transpose attention (STDA) encoder that splits input tensors into multiple channel groups and utilizes depth-wise convolution along with self-attention across channel dimensions to implicitly increase the receptive field and encode multi-scale features. Our extensive experiments on classification, detection and segmentation tasks, reveal the merits of the proposed approach, outperforming state-of-the-art methods with comparatively lower compute requirements. Our EdgeNeXt model with 1.3M parameters achieves 71.2% top-1 accuracy on ImageNet-1K, outperforming MobileViT with an absolute gain of 2.2% with 28% reduction in FLOPs. Further, our EdgeNeXt model with 5.6M parameters achieves 79.4% top-1 accuracy on ImageNet-1K. The code and models are available at https://t.ly/_Vu9.

Original languageEnglish
Title of host publicationComputer Vision – ECCV 2022 Workshops
Subtitle of host publicationTel Aviv, Israel, October 23–27, 2022, Proceedings, Part VII
EditorsLeonid Karlinsky, Tomer Michaeli, Ko Nishino
PublisherSpringer
Pages3-20
Number of pages18
ISBN (Print)978-3-031-25081-1
DOIs
Publication statusPublished - 2023
MoE publication typeA4 Conference publication
EventEuropean Conference on Computer Vision - Tel Aviv, Israel
Duration: 23 Oct 202227 Oct 2022
Conference number: 17
https://eccv2022.ecva.net

Publication series

NameLecture Notes in Computer Science
Volume13807 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceEuropean Conference on Computer Vision
Abbreviated titleECCV
Country/TerritoryIsrael
CityTel Aviv
Period23/10/202227/10/2022
Internet address

Keywords

  • Convolutional neural network
  • Edge devices
  • Hybrid model
  • Image classification
  • Object detection and segmentation
  • Self-attention
  • Transformers

Fingerprint

Dive into the research topics of 'EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications'. Together they form a unique fingerprint.

Cite this