Distributed Inference Acceleration with Adaptive DNN Partitioning and Offloading

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

1 Citation (Scopus)
237 Downloads (Pure)

Abstract

Deep neural networks (DNN) are the de-facto solution behind many intelligent applications of today, ranging from machine translation to autonomous driving. DNNs are accurate but resource-intensive, especially for embedded devices such as mobile phones and smart objects in the Internet of Things. To overcome the related resource constraints, DNN inference is generally offloaded to the edge or to the cloud. This is accomplished by partitioning the DNN and distributing computations at the two different ends. However, most of existing solutions simply split the DNN into two parts, one running locally or at the edge, and the other one in the cloud. In contrast, this article proposes a technique to divide a DNN in multiple partitions that can be processed locally by end devices or offloaded to one or multiple powerful nodes, such as in fog networks. The proposed scheme includes both an adaptive DNN partitioning scheme and a distributed algorithm to offload computations based on a matching game approach. Results obtained by using a self-driving car dataset and several DNN benchmarks show that the proposed solution significantly reduces the total latency for DNN inference compared to other distributed approaches and is 2.6 to 4.2 times faster than the state of the art.

Original languageEnglish
Title of host publicationINFOCOM 2020 - IEEE Conference on Computer Communications
PublisherIEEE
Pages854-863
Number of pages10
ISBN (Electronic)9781728164120
DOIs
Publication statusPublished - Jul 2020
MoE publication typeA4 Article in a conference publication
EventIEEE Conference on Computer Communications - Online, Toronto, Canada
Duration: 6 Jul 20209 Jul 2020
Conference number: 38

Publication series

NameProceedings - IEEE INFOCOM
PublisherIEEE
Volume2020-July
ISSN (Print)0743-166X

Conference

ConferenceIEEE Conference on Computer Communications
Abbreviated titleINFOCOM
CountryCanada
CityToronto
Period06/07/202009/07/2020

Keywords

  • distributed algorithm
  • DNN inference
  • matching game
  • task offloading
  • task partitioning

Fingerprint Dive into the research topics of 'Distributed Inference Acceleration with Adaptive DNN Partitioning and Offloading'. Together they form a unique fingerprint.

Cite this