Attention-Weighted Federated Deep Reinforcement Learning for Device-to-Device Assisted Heterogeneous Collaborative Edge Caching

Xiaofei Wang, Ruibin Li, Chenyang Wang, Xiuhua Li, Tarik Taleb, Victor C.M. Leung

    Research output: Contribution to journalArticleScientificpeer-review

    126 Citations (Scopus)
    852 Downloads (Pure)

    Abstract

    In order to meet the growing demands for multimedia service access and release the pressure of the core network, edge caching and device-to-device (D2D) communication have been regarded as two promising techniques in next generation mobile networks and beyond. However, most existing related studies lack consideration of effective cooperation and adaptability to the dynamic network environments. In this article, based on the flexible trilateral cooperation among user equipment, edge base stations and a cloud server, we propose a D2D-assisted heterogeneous collaborative edge caching framework by jointly optimizing the node selection and cache replacement in mobile networks. We formulate the joint optimization problem as a Markov decision process, and use a deep Q-learning network to solve the long-term mixed integer linear programming problem. We further design an attention-weighted federated deep reinforcement learning (AWFDRL) model that uses federated learning to improve the training efficiency of the Q-learning network by considering the limited computing and storage capacity, and incorporates an attention mechanism to optimize the aggregation weights to avoid the imbalance of local model quality. We prove the convergence of the corresponding algorithm, and present simulation results to show the effectiveness of the proposed AWFDRL framework in reducing average delay of content access, improving hit rate and offloading traffic.

    Original languageEnglish
    Article number9252973
    Pages (from-to)154-169
    Number of pages16
    JournalIEEE Journal on Selected Areas in Communications
    Volume39
    Issue number1
    Early online date2020
    DOIs
    Publication statusPublished - Jan 2021
    MoE publication typeA1 Journal article-refereed

    Keywords

    • attention-weighted federated learning
    • deep reinforcement learning
    • device to device
    • Edge caching

    Fingerprint

    Dive into the research topics of 'Attention-Weighted Federated Deep Reinforcement Learning for Device-to-Device Assisted Heterogeneous Collaborative Edge Caching'. Together they form a unique fingerprint.

    Cite this