loading page

FedCache: A Knowledge Cache-driven Federated Learning Architecture for Personalized Edge Intelligence
  • +6
  • Zhiyuan Wu ,
  • Sheng Sun ,
  • Yuwei Wang ,
  • Min Liu ,
  • Ke Xu ,
  • Wen Wang ,
  • Xuefeng Jiang ,
  • Bo Gao ,
  • Jinda Lu
Zhiyuan Wu
Institute of Computing Technology, Institute of Computing Technology, Institute of Computing Technology

Corresponding Author:[email protected]

Author Profile
Sheng Sun
Author Profile
Yuwei Wang
Author Profile
Xuefeng Jiang
Author Profile

Abstract

Edge Intelligence (EI) enables Artificial Intelligence (AI) applications to run at the edge, where data analysis and decision-making can be performed in real-time and close to data sources. To protect data privacy and unify data silos distributed among end devices in EI, Federated Learning (FL) is proposed for collaborative training shared AI models across multiple devices without compromising data security.  However, the prevailing FL approaches cannot guarantee model generalization and adaptation on heterogeneous clients. Recently, Personalized Federated Learning (PFL) has drawn growing awareness in EI, as it enables striking a productive balance between local-specific training requirements inherent in devices and global-generalized optimization objectives for satisfactory performance.  However, most existing PFL methods are based on the Parameters Interaction-based Architecture (PIA) represented by FedAvg, which causes unaffordable communication burdens due to large-scale parameters transmission between devices and the edge server. In contrast, Logits Interaction-based Architecture (LIA) enables to update model parameters with logits transfer, and gains the advantages of communication lightweight and heterogeneous on-device model allowance compared to PIA. Nevertheless, previous LIA methods attempt to achieve satisfactory performance either relying on unrealistic public datasets or increasing communication overhead for additional information transmission other than logits. To tackle this dilemma, we propose a knowledge cache-driven PFL architecture, named FedCache, which reserves a knowledge cache on the server for fetching personalized knowledge from the samples with similar hashes to each given on-device sample. During the training phase, ensemble distillation is applied to on-device models for constructive optimization with personalized knowledge transferred from the server-side knowledge cache. 
Empirical experiments on four datasets demonstrate the comparable performance of FedCache with state-of-art PFL approaches, with more than two orders of magnitude improvements in communication efficiency. Our code and DEMO are available at https://github.com/wuzhiyuan2000/FedCache.
01 Feb 2024Submitted to TechRxiv
11 Feb 2024Published in TechRxiv