Mitigating Dynamic Graph Distribution Shifts via Mixture of Variational Experts

摘要

Dynamic graph neural networks (DyGNNs) currently struggle with handling distribution shifts that naturally arise when training and test data follow similar but non-identical distributions. As the generation of dynamic graphs is strongly influenced by latent environments, it is critical to investigate their impacts on the generalization behavior of DyGNNs. We therefore establish a connection between the temporal message-passing scheme employed by DyGNNs and their generalization performance under distribution shifts. Our analysis reveals that environment-specific factors misguide the learning process and lead to unsatisfactory out-of-distribution (OOD) generalization. Based on this insight, we propose MoVE, a Mixture of Variational Experts network to mitigate complex distribution shifts in dynamic graphs. MoVE adopts a hierarchical variational architecture that extrapolates latent representations into a mixture of distribution shifts as pseudo-environments. Additionally, we incorporate a Mixture-of-Experts (MoE) framework with a novel training objective that aligns the outputs of different experts to produce invariant representations. Extensive experiments on various dynamic graphs, including both real-world and synthetic datasets, demonstrate that our model significantly outperforms state-of-the-art techniques.