Adaptive Cache Policy Optimization Through Deep Reinforcement Learning in Dynamic Cellular Networks

Ashvin Srinivasan*, Mohsen Amidzade, Junshan Zhang, Olav Tirkkonen

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

19 Downloads (Pure)

Abstract

We explore the use of caching both at the network edge and within User Equipment (UE) to alleviate traffic load of wireless networks. We develop a joint cache placement and delivery policy that maximizes the Quality of Service (QoS) while simultaneously minimizing backhaul load and UE power consumption, in the presence of an unknown time-variant file popularity. With file requests in a time slot being affected by download success in the previous slot, the caching system becomes a non-stationary Partial Observable Markov Decision Process (POMDP). We solve the problem in a deep reinforcement learning framework based on the Advantageous Actor-Critic (A2C) algorithm, comparing Feed Forward Neural Networks (FFNN) with a Long Short-Term Memory (LSTM) approach specifically designed to exploit the correlation of file popularity distribution across time slots. Simulation results show that using LSTM-based A2C outperforms FFNN-based A2C in terms of sample efficiency and optimality, demonstrating superior performance for the non-stationary POMDP problem. For caching at the UEs, we provide a distributed algorithm that reaches the objectives dictated by the agent controlling the network, with minimum energy consumption at the UEs, and minimum communication overhead.

Original languageEnglish
Pages (from-to)81-99
Number of pages19
JournalIntelligent and Converged Networks
Volume5
Issue number2
DOIs
Publication statusPublished - 2024
MoE publication typeA1 Journal article-refereed

Keywords

  • advantageous actor critic
  • deep reinforcement learning
  • long short term memory
  • non-stationary Partial Observable Markov Decision Process (POMDP)
  • wireless caching

Fingerprint

Dive into the research topics of 'Adaptive Cache Policy Optimization Through Deep Reinforcement Learning in Dynamic Cellular Networks'. Together they form a unique fingerprint.

Cite this