Traffic Steering for Cellular-Enabled UAVs: A Federated Deep Reinforcement Learning Approach

Hamed Hellaoui, Bin Yang, Tarik Taleb, Jukka Manner

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference article in proceedingsScientificvertaisarvioitu

21 Lataukset (Pure)

Abstrakti

This paper investigates the fundamental traffic steering issue for cellular-enabled unmanned aerial vehicles (UAVs), where each UAV needs to select one from different Mobile Network Operators (MNOs) to steer its traffic for improving the Quality-of-Service (QoS). To this end, we first formulate the issue as an optimization problem aiming to minimize the maximum outage probabilities of the UAVs. This problem is non-convex and non-linear, which is generally difficult to be solved. We propose a solution based on the framework of deep reinforcement learning (DRL) to solve it, in which we define the environment and the agent elements. Furthermore, to avoid sharing the learned experiences by the UAV in this solution, we further propose a federated deep reinforcement learning (FDRL)-based solution. Specifically, each UAV serves as a distributed agent to train separate model, and is then communicated to a special agent (dubbed coordinator) to aggregate all training models. Moreover, to optimize the aggregation process, we also introduce a FDRL with DRL-based aggregation (DRL2A) approach, in which the coordinator implements a DRL algorithm to learn optimal parameters of the aggregation. We consider deep Q-learning (DQN) algorithm for the distributed agents and Advantage Actor-Critic (A2C) for the coordinator. Simulation results are presented to validate the effectiveness of the proposed approach.

AlkuperäiskieliEnglanti
OtsikkoICC 2023 - IEEE International Conference on Communications
AlaotsikkoSustainable Communications for Renaissance
ToimittajatMichele Zorzi, Meixia Tao, Walid Saad
KustantajaIEEE
Sivut6230-6235
Sivumäärä6
ISBN (elektroninen)978-1-5386-7462-8
DOI - pysyväislinkit
TilaJulkaistu - 2023
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisussa
TapahtumaIEEE International Conference on Communications - Rome, Italia
Kesto: 28 toukok. 20231 kesäk. 2023

Julkaisusarja

NimiIEEE International Conference on Communications
Vuosikerta2023-May
ISSN (painettu)1550-3607

Conference

ConferenceIEEE International Conference on Communications
LyhennettäICC
Maa/AlueItalia
KaupunkiRome
Ajanjakso28/05/202301/06/2023

Sormenjälki

Sukella tutkimusaiheisiin 'Traffic Steering for Cellular-Enabled UAVs: A Federated Deep Reinforcement Learning Approach'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä