The rapid proliferation of multi-view data has necessitated robust and scalable clustering techniques capable of capturing complex, high-dimensional patterns. While Multi-view Bipartite Graph Clustering (MVBGC) has shown promising results, existing approaches often overlook that the generated bipartite graph is susceptible to disturbances from complex structures and noise. To address these challenges, we propose RTGD-MVC, a novel framework for Robust Tensor Learning with Graph Diffusion tailored for efficient and scalable multi-view graph clustering. RTGD-MVC integrates a graph diffusion mechanism to suppress noise propagation and employs cross-view diffusion to enhance global consistency while capturing complementary information across views. Additionally, a non-convex Tensor Exponential Norm (TEN) is introduced as a tighter surrogate for the tensor rank, enabling the learning of more discriminative and noise-robust representations. By embedding these components into a unified optimization model with linear computational complexity, RTGD-MVC achieves both theoretical efficiency and practical scalability. Extensive experiments on diverse benchmark datasets demonstrate that RTGD-MVC significantly outperforms state-of-the-art methods, highlighting its superior ability to capture intricate multi-view correlations and structural patterns.
论文检索
输入标题、作者或关键词，从 9,256 篇学术成果中精准定位
The rapid urbanization process has significantly increased building energy consumption and carbon emissions, making reliable electricity load forecasting crucial for energy management. However, accurate load forecasting faces three key challenges: (1) complex impact of multimodal data, (2) inter-building semantical relationships, and (3) uncertainty modeling of load patterns. To address these, we propose MMLoad, a novel diffusion-based multimodal framework for multi-scenario building load forecasting with three innovations: (i) a Multimodal Data Enhancement Pipeline generating rich building descriptions using LLMs and integrating temporal factors to analyze multimodal impacts; (ii) a Cross-modal Relation Encoder discovering latent interdependencies through hierarchical fusion, projecting buildings into a unified spatio-temporal (ST) embedding space; and (iii) a Scenario-Conditioned Diffusion Generator employing transformer-based denoising with Scenario-Adaptive Normalization (SAN) for diverse trajectory generation with uncertainty quantification. Experiments show MMLoad outperforms state-of-the-art baselines in accuracy while generating plausible future scenarios, establishing a new paradigm for multimodal learning in smart energy systems.
Diffusion models have recently shown strong capabilities in image generation. This paper investigates their potential for semantic segmentation, with a focus on RGB-D tasks that demand precise pixel-level predictions. In particular, we delve into the intermediate activations generated during the reverse Markov step of diffusion process, discovering that these activations can effectively capture the semantic information of an input image, making them outstanding representations for addressing segmentation challenges. This paper proposes Diffusion-Enhanced Multi-Modal Segmenter (DiffuSeg), which innovatively combines RGB features with those generated by an additional diffusion model, facilitating the extraction of comprehensive and nuanced semantic features. Furthermore, we propose the Cross Attention-and-Aggregation Module (CAAM), which not only fosters long-range interactions between RGB and diffusion-derived features but also recalibrates both feature sets before integration, enhancing multi-modal synergy. Additionally, our model incorporates a Dynamic Cascade Kernel (DCK) architecture that exploits local and intricate multi-scale geometric details. As a part of DCK, the Spatial Interaction Module (SIM) dynamically encodes spatial information by establishing pixel-level correlations, thereby enhancing the spatial feature representation capacity. Extensive experiments on two benchmark datasets demonstrate the strong capability of DiffuSeg in handling challenging semantic segmentation tasks.
Recently, since diffusion models show great potential in image generation, many pretrained diffusion models based image composition methods have been proposed for image illumination harmonization. However, they mainly face two key challenges: 1) the effective preservation of foreground appearance (i.e., content structure and texture details, etc); 2) Reasonable generation of the foreground casting shadow. To this end, we propose a novel Image Illumination Harmonization Diffusion model called I 2 HDiffuser to achieve image illumination harmonization with high-fidelity foreground appearance and reasonable cast shadows. I 2 HDiffuser mainly consists of frequency domain feature enhancement branch (FDFEB) and illumination-shadow consistency generation branch (ISCGB). Specifically, FDFEB first introduces the Wavelet Transform Module (WTM) for decomposing composite image features into low-frequency (i.e., illumination features, etc) and high-frequency (i.e., texture and content structure features, etc) components using the Haar wavelet transform. Then the Multi-Condition Guidance Mechanism (M-CGM) is proposed to interact these components as prior conditions, which are further injected into the ISCGB with a noise-to-denoise process for guiding high-fidelity content and background illumination-aware foreground regeneration. Meanwhile, a shadow mask step-wise iterative optimization strategy is introduced to the ISCGB to explicitly provide a reasonable shadow generation space for foreground objects. Extensive experiments on public image harmonization datasets DESOBAv2 and iHarmony4 and real illumination harmonization dataset IH-SG show that the I 2HDiffuser achieves the superiority.
The task of spatiotemporal dynamic modeling of multi-modal high-dimensional neuroimaging data presents a significant challenge in the field of neuroscience. Recent works often integrate attention mechanisms for hierarchical modeling, but the gradual extraction of spatiotemporal features leads to feature isolation. Moreover, attention-based fusion mechanisms (such as cross attention) tend to focus on learning the self-similarity between different modalities, lacking sufficient exploration of the complementary information across modalities. To address these challenges, we propose the cross swin 4D transformer (CrosST), which can efficiently learn the spatiotemporal patterns of multi-modal high-dimensional neuroimaging data in an end-to-end manner. The unique diffusion cross attention fusion mechanism of CrosST connects features from different modalities through a diffusion strategy during the attention computation, enabling the transfer of differential information between modalities and achieving deep fusion of multi-modal coupled features. Additionally, a voxel interaction strategy is employed to alleviate the computational burden during the fusion process. Furthermore, CrosST utilizes a 4D shifted window technique to effectively combine local and global information, and introduces the innovative 4D-Mamba algorithm to enhance computational efficiency. We validate the model using a large-scale Alzheimer's disease dataset and design a multi-granularity cognitive stage task for evaluation. The results demonstrate the effectiveness of CrosST.
Medical image synthesis is crucial in clinical workflows, enabling the generation of missing modalities from available imaging data. While recent diffusion-based models show promise in medical image synthesis, they face two key limitations: progressive distribution drift from coarse intermediate samples and structural granularity loss due to missing high-frequency constraints. To address these challenges, we propose Dual Diffusion Bridge (DualDB), a framework integrating implicit distribution alignment and explicit structural constraints within a unified diffusion bridge paradigm. First, implicit distribution alignment employs optimal transport-guided adversarial learning to minimize statistical discrepancies between intermediate and target distributions, mitigating global distribution drift. Second, explicit structural alignment applies gradient-driven constraints to preserve high-frequency anatomical features, preventing structural degradation during reverse diffusion. This complementary design ensures both global statistical consistency and local anatomical precision in the synthesized results. Extensive experiments on multi-contrast MRI and MRI-CT translation show that DualDB outperforms state-of-the-art methods in quantitative performance and visual fidelity, maintaining superior anatomical accuracy even under noisy conditions.
Recent advances in diffusion models have notably enhanced text-to-image (T2I) generation quality, but they also raise the risk of generating unsafe content. Traditional safety methods like text blacklisting or harmful content classification have significant drawbacks: they can be easily circumvented or require extensive datasets and extra training. To overcome these challenges, we introduce PurifyGen, a novel, training-free approach for safe T2I generation that retains the model's original weights. PurifyGen introduces a dual-stage strategy for prompt purification. First, we evaluate the safety of each token in a prompt by computing its complementary semantic distance, which measures the semantic proximity between the prompt tokens and concept embeddings from predefined toxic and clean lists. This enables fine-grained prompt classification without explicit keyword matching or retraining. Tokens closer to toxic concepts are flagged as risky. Second, for risky prompts, we apply a dual-space transformation: we project toxic-aligned embeddings into the null space of the toxic concept matrix, effectively removing harmful semantic components, and simultaneously align them into the range space of clean concepts. This dual alignment purifies risky prompts by both subtracting unsafe semantics and reinforcing safe ones, while retaining the original intent and coherence. We further define a token-wise strategy to selectively replace only risky token embeddings, ensuring minimal disruption to safe content. PurifyGen offers a plug-and-play solution with theoretical grounding and strong generalization to unseen prompts and models. Extensive testing shows that PurifyGen surpasses current methods in reducing unsafe content across five datasets and competes well with training-dependent approaches.
Traditional Camouflaged Object Detection (COD) methods heavily depend on labor-intensive annotated datasets which require extensive manual effort, resulting in limited generalization. While recent studies have combined Multimodal Large Language Models (MLLMs) and Vision Foundation Models (VFMs) to achieve zero-shot COD, their performance is hindered by modality gap between linguistic semantics and fine-grained visual cues, especially in complex camouflage scenarios. In this paper, we propose Language-to-instance generative visual Prompting (LiP), a novel framework that addresses this limitation by transforming text prompts generated by MLLMs into instance-level visual prompts through a text-to-image generative process. Specifically, we introduce a Diffusion-driven Visual Prompt Generation (DVPG) module that leverages Stable Diffusion model to synthesize visual references, enabling robust homogeneous modality matching for COD. Additionally, we introduce Instruction Contrastive Reasoning (ICR) module to enhance the semantic reliability of prompts by suppressing hallucinated concepts during MLLM inference. To the best of our knowledge, LiP is the first framework that utilize text-to-image generative model to construct instance-level visual prompts in COD task. Extensive experiments on four benchmark datasets demonstrate the effectiveness and strong generalization ability of our approach.
Transparent object segmentation from a single image has been investigated for several years. However, detecting transparent areas from video has not been well explored, especially for different kinds of transparent categories besides glass, due to the scarcity of such a dataset. Therefore, in this paper, we propose the video-based transparent object segmentation task and introduce the first-of-its-kind corresponding dataset named TransVid, which contains nearly 400 videos with a total of 18,523 frames. Based on TranVid, we further propose a new method called TranSeg, in which we innovatively introduce Graph Neural Networks into the temporal segmentation task and combined with a novel Diffusion Model to make the model's segmentation results more accurate. Experimental results show that TranSeg achieves higher accuracy with fewer parameters than previous state-of-the-art models, demonstrating the effectiveness of our method. Moreover, comprehensive ablation analysis reveal several fascinating insights and suggest viable paths for further research.
Recent latent diffusion models (LDMs) have been explored to generate diverse domain-specific images based on source domain data, showing promising performance in domain generalization tasks. However, although the generated images present counterfactual augmentation, such as the background and style changes, the distortion of object details disrupts the causal factors, such as texture and shape. This leads to negative outcomes when directly applying LDM to domain generalization in object detection. To address the problems mentioned above, we propose Object-Preserving Counterfactual Diffusion augmentation method (OPCD) to explore the diffusion model to generate diverse domain-specific images without disrupting the object details. First, we construct a region-aware image generation framework, which leverages labeled source domain data to guide LDM in generating region-constrained images that preserve the semantic consistency of the original source images. Second, we propose object-preserving counterfactual augmentation, which retains the object region of the generated image and fuses diversified global information. This ensures that object details are not distorted and that the generated information is maintained. Third, to reduce the resource burden of generating a large number of images in LDM, we design a random insertion strategy. It mixes generated and source domain images, turning limited diversity samples into abundant training data. Experimental results on several benchmark datasets show that OPCD outperforms existing methods in single-domain generalized object detection. Codes can be found at https://github.com/qinhongda8/OPCD.
Noise Matters: Diffusion Model-based Urban Mobility Generation with Collaborative Noise Priors
PDF ↗With global urbanization, the focus on sustainable cities has largely grown, driving research into equity, resilience, and urban planning, which often relies on mobility data. The rise of web-based apps and mobile devices has provided valuable user data for mobility-related research. However, real-world mobility data is costly and raises privacy concerns. To protect privacy while retaining key features of real-world movement, the demand for synthetic data has steadily increased. Recent advances in diffusion models have shown great potential for mobility trajectory generation due to their ability to model randomness and uncertainty. However, existing approaches often directly apply identically distributed (i.i.d.) noise sampling from image generation techniques, which fail to account for the spatiotemporal correlations and social interactions that shape urban mobility patterns. In this paper, we propose CoDiffMob, a diffusion model for urban mobility generation with collaborative noise priors, we emphasize the critical role of noise in diffusion models for generating mobility data. By leveraging both individual movement characteristics and population-wide dynamics, we construct novel collaborative noise priors that provide richer and more informative guidance throughout the generation process. Extensive experiments demonstrate the superiority of our method, with generated data accurately capturing both individual preferences and collective patterns, achieving an improvement of over 32%. Furthermore, it can effectively replace web-derived mobility data to better support downstream applications, while safeguarding user privacy and fostering a more secure and ethical web. This highlights its tremendous potential for applications in sustainable city-related research. The code and data are available at https://github.com/tsinghua-fib-lab/CoDiffMob.
Nowadays, Graph Fraud Detection (GFD) in financial scenarios has become an urgent research topic to protect online payment security. However, as organized crime groups are becoming more professional in real-world scenarios, fraudsters are employing more sophisticated camouflage strategies. Specifically, fraudsters disguise themselves by mimicking the behavioral data collected by platforms, ensuring that their key characteristics are consistent with those of benign users to a high degree, which we call Adaptive Camouflage. Consequently, this narrows the differences in behavioral traits between them and benign users within the platform's database, thereby making current GFD models lose efficiency. To address this problem, we propose a relation diffusion-based graph augmentation model Grad. In detail, Grad leverages a supervised graph contrastive learning module to enhance the fraud-benign difference and employs a guided relation diffusion generator to generate auxiliary homophilic relations from scratch. Based on these, weak fraudulent signals would be enhanced during the aggregation process, thus being obvious enough to be captured. Extensive experiments have been conducted on two real-world datasets provided by WeChat Pay, one of the largest online payment platforms with billions of users, and three public datasets. The results show that our proposed model Grad outperforms SOTA methods in both various scenarios, achieving at most 11.10% and 43.95% increases in AUC and AP, respectively.
A question-answering (QA) simulator is a model that simulates human students QA behaviors. By leveraging QA history to estimate the probability of correctly answering a newly recommended question, the simulator enables the educational recommender systems to be trained in a simulated environment, protecting human students from the potential negative impact of low-quality recommendations. Despite its significant importance, the construction of QA simulators has not been thoroughly explored in the research domain of AI. Previous methods mainly rely on existing knowledge tracing (KT) models to construct such a simulator. However, due to the discrepancy between the KT task and the simulation task, those KT-based simulators suffer from severe bias accumulation, which limits the effectiveness of the simulation. In this paper, we propose a method called Diffusion-based Simulator (DSim), which takes advantage of diffusion to alleviate the bias accumulation. To our knowledge, DSim is the first to focus on building a QA simulator.
Instruction Vulnerability Prediction for WebAssembly with Semantic Enhanced Code Property Graph
PDF ↗WebAssembly (Wasm) is a universal low-level bytecode designed to build modern web systems. Recent studies have shown that technologies such as voltage scaling and RowHammer attacks are expected to increase the likelihood of bit flips, which may cause unacceptable or catastrophic system failures. This raises concerns about the impact of bit flips on Wasm programs, which run as instructions in web systems, and it is an undeveloped topic since the features of Wasm differ from traditional programs. In this paper, we propose a novel paradigm, namely IVPSEG, to understand the error propagation of bit flips within Wasm programs. Specifically, we first use Large Language Models (LLMs) to automatically extract instruction embeddings containing semantic knowledge of each instruction's context. Then, we exploit these embeddings and program structure (control execution and data transfer) to construct a semantic enhanced code property graph, which implicates the potential path of error propagation. Based on this graph, we utilize graph neural networks and attention diffusion to optimize instruction embeddings by capturing different error propagation patterns for instruction vulnerability prediction. In particular, we build a Wasm compilation and fault generation system to simulate bit flips at Wasm runtime. Our experimental results with 14 benchmark programs and test cases show IVPSEG outperforms the state-of-the-art methods in terms of accuracy (average 13.06%ͽ ), F1-score (average 14.93%↑), and model robustness.
The diffusion of culture online is theorized to be influenced by many interacting social factors (e.g., network and identity). However, most existing computational cascade models consider just a single factor (e.g., network or identity). This work offers a new framework for teasing apart the mechanisms underlying hashtag cascades. We curate a new dataset of 1,337 hashtags representing cultural innovation online, develop a 10-factor evaluation framework for comparing empirical and simulated cascades, and show that a combined network+identity model better simulates hashtag cascades than network- or identity-only counterfactuals. We also explore heterogeneity in performance: While a combined network+identity model best predicts the popularity of cascades, a network-only model best predicts cascade growth and an identity-only model best predicts adopter composition. The network+identity model has the highest comparative advantage among hashtags used for expressing racial or regional identity and talking about sports or news. In fact, we are able to predict what combination of network and/or identity best models each hashtag and use this to further improve performance. Our results show the utility of models incorporating the interactions of network, identity, and other social factors in the diffusion of hashtags in social media.
Text-to-image models based on diffusion processes, such as DALL-E, Stable Diffusion, and Midjourney, are capable of transforming texts into detailed images and have widespread applications in art and design. As such, amateur users can easily imitate professional-level paintings by collecting an artist's work and fine-tuning the model, leading to concerns about artworks' copyright infringement. To tackle these issues, previous studies either add visually imperceptible perturbation to the artwork to change its underlying styles (perturbation-based methods) or embed post-training detectable watermarks in the artwork (watermark-based methods). However, when the artwork or the model has been published online, i.e., modification to the original artwork or model retraining is not feasible, these strategies might not be viable. To this end, we propose a novel method for data-use auditing in the text-to-image generation model. The general idea of ArtistAuditor is to identify if a suspicious model has been finetuned using the artworks of specific artists by analyzing the features related to the style. Concretely, ArtistAuditor employs a style extractor to obtain the multi-granularity style representations and treats artworks as samplings of an artist's style. Then, ArtistAuditor queries a trained discriminator to gain the auditing decisions. The experimental results on six combinations of models and datasets show that ArtistAuditor can achieve high AUC values (> 0.937). By studying ArtistAuditor's transferability and core modules, we provide valuable insights into the practical implementation. Finally, we demonstrate the effectiveness of ArtistAuditor in real-world cases by an online platform Scenario. ArtistAuditor is open-sourced at https://github.com/Jozenn/ArtistAuditor.
Robust Graph Learning Against Adversarial Evasion Attacks via Prior-Free Diffusion-Based Structure Purification
PDF ↗Adversarial evasion attacks pose significant threats to graph learning, with lines of studies that have improved the robustness of Graph Neural Networks (GNNs). However, existing works rely on priors about clean graphs or attacking strategies, which are often heuristic and inconsistent. To achieve robust graph learning over different types of evasion attacks and diverse datasets, we investigate this problem from a prior-free structure purification perspective. Specifically, we propose a novel Diffusion-based Structure Purification framework named DiffSP, which creatively incorporates the graph diffusion model to learn intrinsic distributions of clean graphs and purify the perturbed structures by removing adversaries under the direction of the captured predictive patterns without relying on priors. DiffSP is divided into the forward diffusion process and the reverse denoising process, during which structure purification is achieved. To avoid valuable information loss during the forward process, we propose an LID-driven non-isotropic diffusion mechanism to selectively inject noise anisotropically. To promote semantic alignment between the clean graph and the purified graph generated during the reverse process, we reduce the generation uncertainty by the proposed graph transfer entropy guided denoising mechanism. Extensive experiments demonstrate the superior robustness of DiffSP against evasion attacks.
The distributionally robust optimization (DRO)-based graph neural network methods improve recommendation systems' out-of-distribution (OOD) generalization by optimizing the model's worst-case performance. However, these studies fail to consider the impact of noisy samples in the training data, which results in diminished generalization capabilities and lower accuracy. Through experimental and theoretical analysis, this paper reveals that current DRO-based graph recommendation methods assign greater weight to noise distribution, leading to model parameter learning being dominated by it. When the model overly focuses on fitting noise samples in the training data, it may learn irrelevant or meaningless features that cannot be generalized to OOD data. To address this challenge, we design a Distributionally Robust Graph model for OOD recommendation (DRGO). Specifically, our method first employs a simple and effective diffusion paradigm to alleviate the noisy effect in the latent space. Additionally, an entropy regularization term is introduced in the DRO objective function to avoid extreme sample weights in the worst-case distribution. Finally, we provide a theoretical proof of the generalization error bound of DRGO as well as a theoretical analysis of how our approach mitigates noisy sample effects, which helps to better understand the proposed framework from a theoretical perspective. We conduct extensive experiments on four datasets to evaluate the effectiveness of our framework against three typical distribution shifts, and the results demonstrate its superiority in both independently and identically distributed distributions (IID) and OOD.
Road trajectory generation creates synthetic yet realistic trajectories to tackle data collection costs and privacy concerns. Existing methods generate a trajectory either segment-by-segment using sequence models or holistically in one step using diffusion models. Sequence-based models have good regularity and consistency (i.e., resemble the input trajectories) but lack diversity, while diffusion-based models enhance diversity but sacrifice regularity and consistency. To combine the merits of existing methods, we propose Seed, by bridging sequence and diffusion models for trajectory generation. In particular, Seed adopts a conditional diffusion structure, where a Transformer models the movement of each trajectory along the road segments, and conditioned on the Transformer's output, a diffusion model recovers the next road segment from random noise. The rationale is that the Transformer captures sequential patterns for regularity and consistency, while the diffusion model introduces diversity by recovering from noise. We use a trajectory reconstruction task to train Seed, and design a curriculum learning strategy to accelerate convergence. We compare Seed with 8 state-of-the-art trajectory generation methods on 3 datasets, and the results show that Seed improves the best-performing baseline by over 50%.
Diffusion models (DMs) have emerged as the new state-of-the-art family of deep generative models. To gain deeper insights into the limitations of diffusion models in recommender systems, we investigate the fundamental structural disparities between images and items. Consequently, items often exhibit distinct anisotropic and directional structures that are less prevalent in images. However, the traditional forward diffusion process continuously adds isotropic Gaussian noise, causing anisotropic signals to degrade into noise, which impairs the semantically meaningful representations in recommender systems. Inspired by the advancements in hyperbolic spaces, we propose a novel Hyperbolic Diffusion Recommender Model (named HDRM). Unlike existing directional diffusion methods based on Euclidean space, the intrinsic non-Euclidean structure of hyperbolic space makes it particularly well-adapted for handling anisotropic diffusion processes. In particular, we begin by constructing a geometrically latent space grounded in hyperbolic geometry, incorporating interpretability measures to define the latent anisotropic diffusion processes. Subsequently, we propose a novel hyperbolic latent diffusion process specifically tailored for users and items. Drawing upon the natural geometric attributes of hyperbolic spaces, we restrict both radial and angular components to facilitate directional diffusion propagation, thereby ensuring the preservation of the original topological structure in user-item interaction graphs. Extensive experiments on three benchmark datasets demonstrate the effectiveness of HDRM. Our code is available at https://github.com/yuanmeng-cpu/HDRM.