Traffic Steering for Cellular-Enabled UAVs: A Federated Deep Reinforcement Learning Approach

Hamed Hellaoui, Bin Yang, Tarik Taleb, Jukka Manner

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

18 Downloads (Pure)

Abstract

This paper investigates the fundamental traffic steering issue for cellular-enabled unmanned aerial vehicles (UAVs), where each UAV needs to select one from different Mobile Network Operators (MNOs) to steer its traffic for improving the Quality-of-Service (QoS). To this end, we first formulate the issue as an optimization problem aiming to minimize the maximum outage probabilities of the UAVs. This problem is non-convex and non-linear, which is generally difficult to be solved. We propose a solution based on the framework of deep reinforcement learning (DRL) to solve it, in which we define the environment and the agent elements. Furthermore, to avoid sharing the learned experiences by the UAV in this solution, we further propose a federated deep reinforcement learning (FDRL)-based solution. Specifically, each UAV serves as a distributed agent to train separate model, and is then communicated to a special agent (dubbed coordinator) to aggregate all training models. Moreover, to optimize the aggregation process, we also introduce a FDRL with DRL-based aggregation (DRL2A) approach, in which the coordinator implements a DRL algorithm to learn optimal parameters of the aggregation. We consider deep Q-learning (DQN) algorithm for the distributed agents and Advantage Actor-Critic (A2C) for the coordinator. Simulation results are presented to validate the effectiveness of the proposed approach.

Original languageEnglish
Title of host publicationICC 2023 - IEEE International Conference on Communications
Subtitle of host publicationSustainable Communications for Renaissance
EditorsMichele Zorzi, Meixia Tao, Walid Saad
PublisherIEEE
Pages6230-6235
Number of pages6
ISBN (Electronic)978-1-5386-7462-8
DOIs
Publication statusPublished - 2023
MoE publication typeA4 Conference publication
EventIEEE International Conference on Communications - Rome, Italy
Duration: 28 May 20231 Jun 2023

Publication series

NameIEEE International Conference on Communications
Volume2023-May
ISSN (Print)1550-3607

Conference

ConferenceIEEE International Conference on Communications
Abbreviated titleICC
Country/TerritoryItaly
CityRome
Period28/05/202301/06/2023

Keywords

  • Cellular Networks
  • Connection Steering
  • Deep Reinforcement Learning (DRL)
  • FDRL with DRL-based Aggregation (DRL2A)
  • Federated Deep Reinforcement Learning (FDRL)
  • Unmanned Aerial Vehicles (UAVs)

Fingerprint

Dive into the research topics of 'Traffic Steering for Cellular-Enabled UAVs: A Federated Deep Reinforcement Learning Approach'. Together they form a unique fingerprint.

Cite this