Graph Cross-Domain Continual Fine-Tuning via Orthogonal LoRA Routing with Contrastive Expert Specialization

摘要

This paper investigates a novel and critical problem of Graph Cross-Domain Continual Fine-Tuning, which aims to adapt a large pre-trained Graph Foundation Model across diverse domains. Existing continual graph learning methods are mostly limited to incremental settings within only a single domain, and are typically trained from scratch. As a result, they fail to handle cross-domain shifts effectively, suffer from severe forgetting, and lack transferability. To address these challenges, we present G-CORMoL, Graph Continual Fine-tuning with Orthogonal, Router-driven Mixture of LoRA experts. G-CORMoL achieves effective adaptation while preserving prior knowledge by enforcing mathematical orthogonality between expert LoRA adapters, thereby eliminating interference across tasks. It further supports cross-domain knowledge transfer through a symmetric dual-driven routing mechanism that learns a global composition policy over all learned LoRA experts. In addition, it promotes expert specialization via a contrastive objective with theoretical guarantees. Extensive experiments on different cross-domain task orders demonstrate that G-CORMoL achieves robust state-of-the-art performance, not only preventing catastrophic forgetting but also leveraging accumulated knowledge to enable positive transfer.