FOREX Press I. J. of Electrical & Electronics Research
Support Open Access
  • Home/
  • IJEER-130314

Research Article |

Optimized D2D Mode Selection in H-CRANs Using Asynchronous Federated Deep Reinforcement Learning with Federated Averaging

Author(s): Abhishek Malviya1,2*,Sudhakar Pandey2

Published In : International Journal of Electrical and Electronics Research (IJEER)        Volume 13, Issue 2

Publisher : FOREX Publication

Published : 30 August 2025

e-ISSN : 2347-470X

Page(s) : 493-500

DOI: https://doi.org/10.37391/IJEER.130314

Abstract

In the context of heterogeneous cloud radio access networks (H-CRANs), there's a growing challenge in achieving efficient resource allocation due to the surge in data and complexities related to asynchrony in federated learning frameworks. This research presents an innovative methodology that blends federated averaging with asynchronous federated deep reinforcement learning (AF-DRL). By enabling individual agents to interact with distinct environments and subsequently adjust policy parameters, local updates are consolidated at a central point to generate a global update. Through this iterative procedure, the system attains optimal transmission mode selection and enhanced resource block distribution, thereby significantly boosting the overall performance of H-CRANs while ensuring that latency and reliability benchmarks are met.

Keywords: Heterogeneous Cloud Radio Access Networks (H-CRANs), Resource Allocation, Federated Averaging, Asynchronous Federated Deep Reinforcement Learning (AF-DRL), Communication Efficiency, Federated Learning.




Abhishek Malviya,Department of Computer Science & Engineering, United Institute of Technology, Prayagraj, India; Email: abhishekmalviya2050@gmail.com

Abhishek Malviya , Department of Information Technology, NIT Raipur, Chhattisgarh, India; Email: spandey.it@nitrr.ac.in

Sudhakar Pandey,Department of Information Technology, NIT Raipur, Chhattisgarh, India; Email: spandey.it@nitrr.ac.in

    [1] Yunlong Lu; Xiaohong Huang; Ke Zhang; Sabita Maharjan; Yan Zhang; “Blockchain Empowered Asynchronous Federated Learning for Secure Data Sharing in Internet of Vehicles”, IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, 2020.
    [2] Xinran Zhang; Mugen Peng; Shi Yan; Yaohua Sun; “Deep-Reinforcement- Learning-Based Mode Selection and Resource Allocation for Cellular V2X Communications”, IEEE INTERNET OF THINGS JOURNAL,2020.
    [3] Xinran Zhang; Mugen Peng; Shi Yan; Yaohua Sun; “Deep Reinforcement Learning Based Mode Selection and Resource Allocation for Cellular V2X Communications”, ARXIV-CS.NI, 2020.
    [4] H.Wang; Z. Kaplan; D. Niuand B. Li; “Optimizing Federated Learning on Non-IID Data with Reinforcement Learning”, INFOCOM, 2020.
    [5] Junjie Tan; Ying-Chang Liang; Lin Zhang; Gang Feng; “Deep Reinforcement Learning for Joint Channel Selection and Power Control in D2D Networks”, IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, 2021.
    [6] XueqingZhang; Yanwei Liu; JinxiaLiu; Antonios Argyriou; Yanni Han; “D2D- Assisted Federated Learning in Mobile Edge Computing Networks”, 2021 IEEE WIRELESS COMMUNICATIONS AND NETWORKING CONFERENCE..., 2021.
    [7] Yunlong Lu; Xiaohong Huang; Ke Zhang; Sabita Maharjan; Yan Zhang; “Communication-Efficient Federated Learning and Permissioned Blockchain for Digital Twin Edge Networks”, IEEE INTERNET OF THINGS JOURNAL,2021.
    [8] Xiaofei Wang; Ruibin Li;Chenyang Wang; Xiuhua Li; Tarik Taleb; VictorC.M.Leung; “Attention-Weighted Federated Deep Reinforcement Learning for Device-to-Device Assisted Heterogeneous Collaborative Edge Caching”,IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS,2021.
    [9] Peiying Zhang; Chao Wang; Chunxiao Jiang; Zhu Han;”Deep Reinforcement Learning Assisted Federated Learning Algorithm for Data Management of IIoT”, IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS,2021.
    [10] Rong Yu; Peichun Li; “Toward Resource-Efficient Federated Learning in Mobile Edge Computing”, IEEE NETWORK, 2021.
    [11] Qiong Wu; Yu Zhao; Qiang Fan; Pingyi Fan; Jiangzhou Wang; Cui Zhang; “Mobility-Aware Cooperative Cachingin Vehicular Edge Computing Based on Asynchronous Federated and Deep Reinforcement Learning”, ARXIV-CS.DC, 2022.
    [12] WeiYang; WeiXiang; YuanYang; PengCheng; “Optimizing Federated Learning with Deep Reinforcement Learning for Digital Twin Empowered Industrial IoT”, IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS,2023.
    [13] QiongWu; SiyuanWang; PingyiFan; QiangFan; “Deep Reinforcement Learning Based Vehicle Selection for Asynchronous Federated Learning Enabled Vehicular Edge Computing”, ARXIV-CS.LG, 2023.
    [14] Farhad Rezazadeh; Lanfranco Zanzi; Francesco Devoti; Hatim Chergui; Xavier Costa-Perez; Christos Verikoukis: “On the Specialization of FDRL Agents for Scalable and Distributed 6G RAN Slicing Orchestration”, ARXIV-CS.NI, 2022.
    [15] Shenglai Zeng; ZonghangLi; HongfangYu; Zhihao Zhang; Long Luo; BoLi; DusitNiyato; “HFedMS: Heterogeneous Federated Learning with Memorable Data Semantics in Industrial Metaverse”, ARXIV-CS. LG,2022.
    [16] Jianfeng Yang; Jianling Hu; Tianqi Yu; “Federated AI-Enabled In-Vehicle Network Intrusion Detection for Internet of Vehicles”, ELECTRONICS,2022.
    [17] Linlin You; Sheng Liu; Yi Chang; C.Yuen; “ATriple-Step Asynchronous Federated Learning Mechanism for Client Activation, Interaction Optimization, and Aggregation Enhancement, IEEE INTERNET OF THINGS JOURNAL,2022.
    [18] Qiong Wu; Xu Chen; Tao Ouyang; Zhi Zhou; Xiaoxi Zhang; Shusen Yang; Junshan Zhang; “HiFlash: Communication-Efficient Hierarchical Federated Learning with AdaptiveStalenessControlandHeterogeneity-awareClient-EdgeAssociation”, ARXIV-CS.NI, 2023.
    [19] WeiYang; WeiXiang; YuanYang; PengCheng; “Optimizing Federated Learning with Deep Reinforcement Learning for Digital Twin Empowered Industrial IoT”, IEEETRANSACTIONS ON INDUSTRIAL INFORMATICS,2023.
    [20] YangXu; MdZakirul AlamBhuiyan; TianWang; XiaokangZhou; Ashutosh KumarSingh; “C-FDRL:Context-Aware Privacy-Preserving Offloading Through Federated Deep Reinforcement Learning in Cloud-Enabled IoT”, IEEE TRANS- ACTIONS ON INDUSTRIAL INFORMATICS, 2023.
    [21] BinJiang; Jianqiang Li; Huihui Wang; H.Song; “Privacy-Preserving Federated Learning for Industrial Edge Computing Via Hybrid Differential Privacy and Adaptive Compression”, IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, 2023.
    [22] Abdallah Jarwan; M. Ibnkahla; “Edge-Based Federated Deep Reinforcement Learning for IoT Traffic Management”, IEEE INTERNET OF THINGS JOURNAL, 2023.
    [23] Qiong Wu; Siyuan Wang; Pingyi Fan; Qiang Fan; “Deep Reinforcement Learning Based Vehicle Selection for Asynchronous Federated Learning Enabled Vehicular Edge Computing”, ARXIV-CS.LG, 2023.

Abishek Malviya and Sudhakar Pandey(2025),Optimized D2D Mode Selection in H-CRANs Using Asynchronous Federated Deep Reinforcement Learning with Federated Averaging. IJEER 13(3), 493-500. DOI: 10.37391/IJEER.130314.