SDMG: Smoothing Your Diffusion Models for Powerful Graph Representation Learning
Technical University of Berlin · University of Illinois at Chicago · Northwest Polytechnical University Xi'an · Northwestern Polytechnical University, Xi'an · Department of Computer Science, Humboldt University Berlin, Humboldt Universität Berlin · UIC · Humboldt Universität Berlin · Potsdam Institute for Climate Impact Research
PDF 由论文原始站点提供，PaperCompass 不保存论文文件。
摘要
Diffusion probabilistic models (DPMs) have recently demonstrated impressive generative capabilities. There is emerging evidence that their sample reconstruction ability can yield meaningful representations for recognition tasks. In this paper, we demonstrate that the objectives underlying generation and representation learning are not perfectly aligned. Through a spectral analysis, we find that minimizing the mean squared error (MSE) between the original graph and its reconstructed counterpart does not necessarily optimize representations for downstream tasks. Instead, focusing on reconstructing a small subset of features, specifically those capturing global information, proves to be more effective for learning powerful representations. Motivated by these insights, we propose a novel framework, the Smooth Diffusion Model for Graphs (SDMG), which introduces a multi-scale smoothing loss and low-frequency information encoders to promote the recovery of global, low-frequency details, while suppressing irrelevant high-frequency noise. Extensive experiments validate the effectiveness of our method, suggesting a promising direction for advancing diffusion models in graph representation learning.