Client-Server Co-design with Multi-modal Codebooks Makes Better and Faster Federate Knowledge Sharing

摘要

Knowledge graphs (KGs) are widely used to store multi-source and heterogeneous structural knowledge, making federated knowledge graph completion (FedKGC) a crucial research topic. FedKGC aims to complete distributed KGs while maintaining privacy and security. Existing FedKGC methods primarily rely on uni-modal structural embedding aggregation for global knowledge sharing, which suffers from the demanding assumption that intersecting entities exist across different clients and are known by the omniscient server. Meanwhile, these uni-modal structure-only methods neglect the exploitation of client-side multi-modal information. In this paper, we propose a new framework MuCo2 to kill two birds with one stone and facilitate client-server co-design through multi-modal codebooks (MuCo). Moving beyond the traditional structure-only paradigm, we introduce multi-modal information of entities as the foundation for KGC modeling and communication. We design a MuCo-based fine-grained KGC model on the client and a MuCo-based communication mechanism on the server, which does not require entity mapping in global aggregation anymore. Comprehensive experiments demonstrate the effectiveness, generalization, reasonability, efficiency, and explainability of MuCo2.