Federated Multi Agent Deep Reinforcement Learning for Optimized Design of Future Wireless Networks

Federated Multi-Agent Deep Reinforcement Learning (F-MADRL) has been recently attracting increasing research interests, as it offers efficient solutions towards meeting the extreme requirements of Beyond 5G (B5G) and 6G applications. By contrast to centralized Deep Reinforcement Learning (DRL) and Multi-Agent DRL (MADRL), F-MADRL enables edge devices to cooperate without sharing their private data, while reducing the delays and signaling costs inherent to centralized approaches. In this article, we explore the new opportunities brought by F-MADRL by conducting a holistic survey on its related recent works. Firstly, we categorize state-of-the-art F-MADRL approaches, based on some distinctive features such as incurred signaling overhead, privacy level, and aggregation frequency. To better illustrate the behavior and advantages of F-MADRL, it is numerically compared to its centralized and distributed DRL counterparts, through a Sub-6GHz/mmWave band association optimization problem for IoT short packet communications. Finally, we identify and discuss the open research directions and challenges, in order to spur further interests in this promising area.