Thorki: Decoupling General and Personalized Knowledge with Collaborative Fusion for Personalized Federated Learning

摘要

Federated Learning (FL) enables knowledge sharing across client models with privacy preservation. A key challenge in FL is to aggregate general knowledge across clients while preserving personalized knowledge for each client. We discover that this challenge primarily stems from the coupling of general and personalized knowledge across different model layers. Existing methods either fail to fully decouple these two types of knowledge or do not leverage them effectively for inference, leading to suboptimal performance. This paper presents Thorki, a new FL system that decouples general and personalized knowledge throughout all model layers. In Thorki, instead of one model, each client stores its personalized knowledge in a small model and general knowledge in a large model. When making an inference, it employs a fusion network to combine the outputs of the two models, leveraging both types of knowledge adaptively. This new model collaboration paradigm allows clients to share their general knowledge in a federated manner without compromising their personalized inference capabilities. Extensive experiments with three models on four datasets demonstrate that Thorki outperforms state-of-the-art FL systems by 0.16%-19.14% in model accuracy and improves time-to-accuracy performance by 2.00x-10.00x.