FlowerTune: A Cross-Domain Benchmark for Federated Fine-Tuning of Large Language Models
Flower Labs; University of Cambridge · ethicalabs.ai · Samsung AI · Flower Labs · University of Cambridge, Zhejiang University · Zhejiang University · Penn State University · Alibaba Group · University of Nevada, Reno · Gachon University · University of Auckland · University of Melbourne · University of Adelaide · Australian National University · University of Oxford · Sony AI · University of Cambridge
PDF 由论文原始站点提供，PaperCompass 不保存论文文件。
摘要
Large Language Models (LLMs) have achieved state-of-the-art results across diverse domains, yet their development remains reliant on vast amounts of publicly available data, raising concerns about data scarcity and the lack of access to domain-specific, sensitive information. Federated Learning (FL) presents a compelling framework to address these challenges by enabling decentralized fine-tuning on pre-trained LLMs without sharing raw data. However, the compatibility and performance of pre-trained LLMs in FL settings remain largely under explored. We introduce the FlowerTune LLM Leaderboard, a first-of-its-kind benchmarking suite designed to evaluate federated fine-tuning of LLMs across four diverse domains: general NLP, finance, medical, and coding. Each domain includes federated instruction-tuning datasets and domain-specific evaluation metrics. Our results, obtained through a collaborative, open-source and community-driven approach, provide the first comprehensive comparison across 26 pre-trained LLMs with different aggregation and fine-tuning strategies under federated settings, offering actionable insights into model performance, resource constraints, and domain adaptation. This work lays the foundation for developing privacy-preserving, domain-specialized LLMs for real-world applications.