The remarkable ability of diffusion models to generate high-fidelity images has led to their widespread adoption. However, concerns have also arisen regarding their potential to produce Not Safe for Work (NSFW) content and exhibit social biases, hindering their practical use in real-world applications. In response to this challenge, prior work has focused on employing security filters to identify and exclude toxic text, or alternatively, fine-tuning pre-trained diffusion models to erase sensitive concepts. Unfortunately, existing methods struggle to achieve satisfactory performance in the sense that they can have a significant impact on the normal model output while still failing to prevent the generation of harmful content in some cases. In this paper, we propose a novel self-discovery approach to identifying a semantic direction vector in the embedding space to restrict text embedding within a safe region. Our method circumvents the need for correcting individual words within the input text and steers the entire text prompt towards a safe region in the embedding space, thereby enhancing model robustness against all possibly unsafe prompts. In addition, we employ Low-Rank Adaptation (LoRA) for semantic direction vector initialization to reduce the impact on the model performance for other semantics. Furthermore, our method can also be integrated with existing methods to improve their social responsibility. Extensive experiments on benchmark datasets demonstrate that our method can effectively reduce NSFW content and mitigate social bias generated by diffusion models compared to several state-of-the-art baselines. WARNING:This paper contains model-generated images that may be potentially offensive.
论文检索
输入标题、作者或关键词，从 9,256 篇学术成果中精准定位
Diffusion models (DMs) have emerged as promising approaches for sequential recommendation due to their strong ability to model data distributions and generate high-quality items. Existing work typically adds noise to the next item and progressively denoises it guided by the user's interaction sequence, generating items that closely align with user interests. However, we identify two key issues in this paradigm. First, the sequences are often heterogeneous in length and content, exhibiting noise due to stochastic user behaviors. Using such sequences as guidance may hinder DMs from accurately understanding user interests. Second, DMs are prone to data bias and tend to generate only the popular items that dominate the training dataset, thus failing to meet the personalized needs of different users. To address these issues, we propose Distinguished Quantized Guidance for Diffusion-based Sequence Recommendation (DiQDiff), which aims to extract robust guidance to understand user interests and generate distinguished items for personalized user interests within DMs. To extract robust guidance, DiQDiff introduces Semantic Vector Quantization (SVQ) to quantize sequences into semantic vectors (e.g., collaborative signals and category interests) using a codebook, which can enrich the guidance to better understand user interests. To generate distinguished items, DiQDiff personalizes the generation through Contrastive Discrepancy Maximization (CDM), which maximizes the distance between denoising trajectories using contrastive loss to prevent biased generation for different users. Extensive experiments are conducted to compare DiQDiff with multiple baseline models across four widely-used datasets. The superior recommendation performance of DiQDiff against leading approaches demonstrates its effectiveness in sequential recommendation tasks.
Model-Agnostic Social Network Refinement with Diffusion Models for Robust Social Recommendation
PDF ↗Social recommendations (SRs) aim to enhance preference modeling by integrating social networks. However, their effectiveness is mainly constrained by two factors: the noisy social connections that may not reflect shared interests, and the limited number of social connections for most users, which hampers the system's ability to fully leverage social influence. Therefore, it is essential to perform social network refinement by removing noisy connections and adding meaningful ones for robust SRs. Inspired by the denoising capability of generative diffusion models, we propose a Model-Agnostic Social Network Refinement framework with Diffusion Models for Robust Social Recommendation (ARD-SR). Specifically, in the forward process, we corrupt the social network by progressively adding position-specific Gaussian noise calibrated to the user preference similarity, better simulating how the social network responds to noise perturbations. The reverse process learns to denoise, guided by each user's neighborhood preferences from the SR backbone, generating a tailored social network aligned with each user's preference for establishing connections. For effective learning, we design a curriculum-based training mechanism that progressively introduces challenging samples characterized by high sparsity or high noise levels. Finally, ARD-SR and the SR backbone are alternately trained, ensuring a continuous mutual enhancement between the social network refinement and the backbone's user representation learning. To further enhance the quality of the refined social network, (1) we introduce a preference-guided flip operation during inference to improve the input quality; and (2) we modify social connections based on the exponential weighted moving average of ARD-SR's predictions across epochs to reduce fluctuations. Experiments on three datasets show that ARD-SR significantly improves SR performance across multiple SR backbones. The code is released at https://github.com/sunyc123r/ARD-SR.
Graph-based social recommender systems utilize user-item interaction graphs and user-user social graphs to model user preferences. However, their performance can be limited by redundant and noisy information in these two graphs. Although several recommender studies on data denoising exist, most either rely on heuristic assumptions, which limit their adaptability, or use a single model that combines denoising and recommendation, potentially imposing substantial demands on the model capacity. To address these issues, we propose a dual Graph Denoising Social Recommender (GDSR), which consists of two steps: graph denoising and user preference prediction. First, we design a denoising module which exploits a dual diffusion model to alleviate noises in the interaction and social graphs by performing multi-step noise diffusion and removal. We develop three kinds of conditions to guide our dual graph diffusion paradigm and propose a cross-domain signal guidance mechanism to enhance the structure of denoised graphs. Second, we devise a recommender module that employs a dual graph learning structure on denoised graphs to generate recommendations. Moreover, we use additional supervision signals from the diffusion-enhanced data augmentation to introduce a graph contrastive learning task, enhancing the recommender module's representation quality and robustness. Experiment results show the effectiveness of our GDSR.
Graph Neural Networks (GNNs)-based recommendation algorithms typically assume that training and testing data are drawn from independent and identically distributed (IID) spaces. However, this assumption often fails in the presence of out-of-distribution (OOD) data, resulting in significant performance degradation. In this study, we construct a Structural Causal Model (SCM) to analyze interaction data, revealing that environmental confounders (e.g., the COVID-19 pandemic) lead to unstable correlations in GNN-based models, thus impairing their generalization to OOD data. To address this issue, we propose a novel approach, graph representation learning via causal diffusion (CausalDiffRec) for OOD recommendation. This method enhances the model's generalization on OOD data by eliminating environmental confounding factors and learning invariant graph representations. Specifically, we use backdoor adjustment and variational inference to infer the real environmental distribution, thereby eliminating the impact of environmental confounders. This inferred distribution is then used as prior knowledge to guide the representation learning in the reverse phase of the diffusion process to learn the invariant representation. In addition,we provide a theoretical derivation that proves optimizing the objective function of CausalDiffRec can encourage the model to learn environment-invariant graph representations, thereby achieving excellent generalization performance in recommendations under distribution shifts. Our extensive experiments validate the effectiveness of CausalDiffRec in improving the generalization of OOD data, and the average improvement is up to 10.69% on Food, 18.83% on KuaiRec, 22.41% on Yelp2018, and 11.65% on Douban datasets.
Unleashing the Potential of Two-Tower Models: Diffusion-Based Cross-Interaction for Large-Scale Matching
PDF ↗Two-tower models are widely adopted in the industrial-scale matching stage across a broad range of application domains, such as content recommendations, advertisement systems, and search engines. This model efficiently handles large-scale candidate item screening by separating user and item representations. However, the decoupling network also leads to a neglect of potential information interaction between the user and item representations. Current state-of-the-art (SOTA) approaches include adding a shallow fully connected layer(i.e., COLD), which is limited by performance and can only be used in the ranking stage. For performance considerations, another approach attempts to capture historical positive interaction information from the other tower by regarding them as the input features(i.e., DAT). Later research showed that the gains achieved by this method are still limited because of lacking the guidance on the next user intent. To address the aforementioned challenges, we propose a "cross-interaction decoupling architecture" within our matching paradigm. This user-tower architecture leverages a diffusion module to reconstruct the next positive intention representation and employs a mixed-attention module to facilitate comprehensive cross-interaction. During the next positive intention generation, we further enhance the accuracy of its reconstruction by explicitly extracting the temporal drift within user behavior sequences. Experiments on two real-world datasets and one industrial dataset demonstrate that our method outperforms the SOTA two-tower models significantly, and our diffusion approach outperforms other generative models in reconstructing item representations.
Recommender systems (RSs) have become essential for alleviating information overload and matching users with relevant content. Traditionally, RSs have focused on personalized content distribution, leveraging user interaction data and various features to rank and recommend existing items. Recently, diffusion models (DMs) have emerged as powerful generative paradigms, introducing new possibilities for RSs to not only enhance their performance for content distribution but also extend their capability boundaries to personalized content creation. On the one hand, DMs enhance the recommendation performance by mitigating challenges such as sparse user-item interactions, weak latent representations, and noisy data. On the other hand, DMs enable personalized content creation, transforming RSs from passive distributors into active generators of user-specific media assets, such as customized images, posters, and multimedia content. Given such a transformative paradigm shift, this survey provides a comprehensive review of the integration of diffusion models into recommender systems, exploring key methodologies, application scenarios, and their impact on recommendation effectiveness, diversity, and personalization. We categorize DM-based recommendation paradigms into content distribution and content creation, compare integration strategies, and discuss open challenges and future directions. By bridging the gap between diffusion models and recommender systems, this work aims to guide researchers and practitioners in developing the next generation of generative AI-powered recommendation solutions.
High Click-Through Rate (CTR) imagery has proven commercial value for food delivery platforms, driving a need for strategies to generate visually compelling images. Our investigations reveal a positive correlation between appropriate food backgrounds and subsequent user engagement. Despite advancements in diffusion models, inpainting new backgrounds does not guarantee high CTR, and fine-tuning diffusion models for this purpose is prohibitively expensive for the fast-paced online food delivery advertising sector. Consequently, there is a lack of cost-effective, transferable generation frameworks tailored to high-CTR food images. In this paper, we propose FoRAGe, a novel high-CTR Food image Retrieval-Augmented Generation pipeline leveraging ControlNet based on Stable Diffusion. Specifically, we construct a comprehensive food image database encompassing a diverse range of background environments. During image generation, FoRAGe retrieves high-quality background exemplars featuring analogous food subjects from the database and employs the retrieved backgrounds as conditions to guide image synthesis via the ControlNet model. Subsequently, a multimodal CTR prediction model is utilized to identify and select optimal images for deployment. Extensive online experiments demonstrate a significant increase in CTR for images generated by our proposed pipeline, and ablation studies further elucidate the impact of different strategies and configurations. Code is available at https://github.com/jiaxu-feng/FoRAGe.
Mobile traffic forecasting allows operators to anticipate network dynamics and performance in advance, offering substantial potential for enhancing service quality and improving user experience. It involves multiple tasks, including long-term prediction, short-term prediction, and generation tasks that do not rely on historical data. By leveraging the different types of mobile network data generated from these tasks, operators can perform a variety of network optimizations and planning activities, such as base station (BS) deployment, resource allocation, energy optimization, etc. However, existing models are often designed for specific tasks and trained with specialized data, and there is a lack of universal models for traffic forecasting across different urban environments. In this paper, we propose a Universal model for Mobile traffic forecasting (UoMo), aiming to handle diverse forecasting tasks of short/long-term predictions and distribution generation across multiple cities to support network planning and optimization. UoMo combines diffusion models and transformers, where various spatio-temporal masks are proposed to enable UoMo to learn intrinsic features of different tasks, and a contrastive learning strategy is developed to capture the correlations between mobile traffic and urban contexts, thereby improving its transfer learning capability. Extensive evaluations on 9 real-world datasets demonstrate that UoMo outperforms current models in various forecasting tasks and zero/few-shot learning. It shows an average accuracy improvement of 27.85%, 18.57%, and 15.6% in long-term prediction, short-term prediction, and generation tasks, respectively, showcasing its strong forecasting capability. We deploy UoMo on China Mobile's JiuTian platform, leveraging the predicted mobile data to optimize live networks. This optimization includes BS deployment, resulting in a 25.3% increase in served users, and BS sleep control, which reduces equipment depreciation by 40.7%. The source code is available online: https://github.com/tsinghua-fib-lab/UoMo.
MedDiTPro: A Prompt-Guided Diffusion Transformer for Multimodal Longitudinal Medical Data Synthesis
PDF ↗Diffusion models have recently emerged as a state-of-the-art approach for synthetic Electronic Health Record (EHR) generation, offering superior fidelity and diversity over traditional generative models. However, existing diffusion-based methods struggle with unique challenges: limited representation learning and modality utilization, where they fail to explicitly capture inter-modality dependencies and fine-grained code-level interactions, and constrained adaptability due to reliance on U-Net-based architectures, which are not well-suited for handling the heterogeneous and evolving nature of EHR data. Furthermore, current evaluation paradigms rely on either perplexity-based sequence modeling or global distributional measures, lacking robustness in assessing both intra-visit code relationships and inter-visit temporal patterns. To address these limitations, we propose MedDiTPro, a diffusion transformer-based framework that enhances multimodal EHR generation by integrating structured modality-aware guidance. Through a unified transformer for intra-visit representation learning, a modality-specific and datawise prompt learner, and a diffusion transformer with structured guidance, MedDiTPro achieves state-of-the-art performance in generating diverse and clinically meaningful synthetic records. Extensive experiments on publicly available datasets demonstrate that MedDiTPro achieves state-of-the-art fidelity, privacy preservation, and utility.
Diffmv: A Unified Diffusion Framework for Healthcare Predictions with Random Missing Views and View Laziness
PDF ↗Advanced healthcare predictions offer significant improvements in patient outcomes by leveraging predictive analytics. Existing works primarily utilize various views of Electronic Health Record (EHR) data, such as diagnoses, lab tests, or clinical notes, for model training. These methods typically assume the availability of complete EHR views and that the designed model could fully leverage the potential of each view. However, in practice, random missing views and view laziness present two significant challenges that hinder further improvements in multi-view utilization. To address these challenges, we introduce Diffmv, an innovative diffusion-based generative framework designed to advance the exploitation of multiple views of EHR data. Specifically, to address random missing views, we integrate various views of EHR data into a unified diffusion-denoising framework, enriched with diverse contextual conditions to facilitate progressive alignment and view transformation. To mitigate view laziness, we propose a novel reweighting strategy that assesses the relative advantages of each view, promoting a balanced utilization of various data views within the model. Our proposed strategy achieves superior performance across multiple health prediction tasks derived from three popular datasets, including multi-view and multi-modality scenarios.
Single-domain generalization (SDG) in time series classification (TSC) poses significant challenges for current time-series domain generalization methods due to the extremely limited data available from only one source domain. In this study, we propose Segment-dErived Expansion of Domains (SEED), a diffusion-based method that effectively expands domain diversity for SDG. We reveal that individual instances exhibit intrinsic temporal shifts over time, which provides a principled foundation for creating multiple pseudo domains by segmenting each instance into distinct parts. To do so, SEED extracts two complementary representations from each time-series segment: 1) a segment-specific representation that captures diverse distributional variations, and 2) a segment-invariant representation that preserves class semantics. SEED formulates these representations as pseudo-domain prompts to guide a diffusion model in generating diverse yet semantically consistent time-series data. Additionally, SEED introduces a novel prompt-fused sampling for diffusion, enabling flexible recombination of segment-specific features to continuously expand the pseudo-domain space. We provide both theoretical analysis and extensive empirical evaluations on four widely used TSC benchmarks to validate its ability in reducing generalization error and improving model's performances in SDG. In our experiments, SEED significantly improves classification accuracy by 7.68% on average compared to the strong baselines.
Calibrating Pre-trained Language Classifiers on LLM-generated Noisy Labels via Iterative Refinement
PDF ↗The traditional process of creating labeled datasets is labor-intensive and expensive. Recent breakthroughs in open-source large language models (LLMs) have opened up a new avenue in generating labeled datasets automatically for various natural language processing (NLP) tasks, providing an alternative to such an expensive annotation process. However, the reliability of such auto-generated labels remains a significant concern due to inherent inaccuracies. When learning from noisy labels, the model's generalization is likely to be harmed as it is prone to overfit to those label noises. While previous studies in learning from noisy labels mainly focus on synthetic noise and real-world noise, LLM-generated label noise receives less attention. In this paper, we propose SiDyP: Simplex Label Diffusion with Dynamic Prior to calibrate the classifier's prediction, thus enhancing its robustness towards LLM-generated noisy labels. SiDyP retrieves potential true label candidates by neighborhood label distribution in text embedding space and iteratively refines noisy candidates using a simplex diffusion model. Our framework can increase the performance of the BERT classifier fine-tuned on both zero-shot and few-shot LLM-generated noisy label datasets by an average of 7.21% and 7.30% respectively. We demonstrate the effectiveness of SiDyP by conducting extensive benchmarking for different LLMs over a variety of NLP tasks. Our code is available on GitHub: https://github.com/gtfintechlab/SiDyP
Temporal graphs, widely used in social network modeling, are valuable for research but pose challenges due to data accessibility and privacy concerns. High-quality graph generation models can produce surrogate data for sharing and training, benefiting tasks such as behavior analysis, anomaly detection, and data augmentation. However, existing deep learning and probabilistic approaches often struggle to balance global statistical properties with local structural details. To overcome this limitation, we leverage motifs, small subgraphs that serve as the building blocks of complex networks, to encode local information. Based on a spectral analysis of motifs, we propose MoDiff, a novel motif-aware diffusion model for temporal graph generation. MoDiff integrates motifs into a diffusion framework by employing motif-enhanced Hermitian matrices that capture local structures and edge orientations, while the spectral diffusion model efficiently generates graphs. Moreover, MoDiff supports controllable graph generation by adjusting density parameters to simulate the evolution of temporal graphs. Experimental results demonstrate that MoDiff outperforms existing approaches, reducing degree discrepancies by 10-50% and clustering discrepancies by 50-90%, while better preserving higher-order structural features. Our code is available at: https://github.com/Yuwe1XU/MoDiff.
In this paper, we investigate the diffusion containment problem through a novel hyperparametric influence minimization model. This model integrates a hyperparametric diffusion framework into the classical influence minimization paradigm, enabling practical, flexible, and fine-grained control over diffusion dynamics via feature interventions on nodes. The objective is to minimize the diffusion from initial seeds, by optimizing the interventions on node feature values. We analyze the challenges and intrinsic properties of hyperparametric influence minimization and derive an upper-bound on the spread, which quantifies the total uncertainty of nodes remaining inactive during the diffusion process. We prove that it exhibits supermodularity in the context of the node selection problem. Based on that, we further design greedy-based algorithms to solve the problem, which outperform the state-of-the-art methods.
The recently proposed Bayesian Flow Networks (BFNs) show great potential in modeling parameter spaces via a diffusion process, offering a unified strategy for handling continuous, discrete data. However, these parameter diffusion models cannot learn high-level semantic representation from the parameter space since common encoders, which encode data into one static representation, can- not capture semantic changes in parameters. This motivates a new direction: learning semantic representations hidden in the param- eter spaces to characterize noisy data. Accordingly, we propose a representation learning framework named SepDiff which operates in the parameter space to obtain parameter-wise latent semantics that exhibit progressive structures. Specifically, SepDiff proposes a self-encoder to learn latent semantics directly from parameters, rather than from observations. The encoder is then integrated into parameter diffusion model, enabling representation learning with various formats of observations. Mutual information terms further promote the disentanglement of latent semantics and capture mean- ingful semantics simultaneously. We illustrate seven representation learning tasks in SepDiff via expanding this parameter diffusion model, and extensive quantitative experimental results demonstrate the superior effectiveness of SepDiff in learning parameter repre- sentation.
Cross-domain recommendation (CDR) has gained significant attention for its ability to address data sparsity issue. However, most existing CDR methods focus primarily on improving recommendation accuracy while largely overlooking fairness considerations, which can lead to biased outcomes and unfair treatment of different user groups. To solve this critical problem, we investigate whether fairness can be transferred from the source domain to the target domain. Our analysis suggests that fairness can be effectively transferred if the fairness of the source domain is ensured and the distributions of the source and target domains are well aligned. Based on this, we propose the FairCDR, a novel framework that can achieve the knowledge transfer of fairness and user preferences simultaneously. FairCDR owns two phases: single-domain fairness guarantee and inter-domain distribution alignment. In the first phase, we employ an adversarial learning-based recommender (ALR) to disentangle user preferences from sensitive attributes in the source domain. In the second phase, we introduce a new mutual learning-based diffusion model (MLDiff), which engages in mutual learning with ALR to progressively align the distributions of the source and target domains. This improves ALR's adaptability to distribution shifts, ultimately ensuring fairness and recommendation performance in the target domain. Extensive experiments on multiple real-world cross-domain datasets demonstrate that FairCDR surpasses existing strong baselines in both fairness and recommendation quality.
Modality-Aware Diffusion Augmentation with Consistent Subspace Disentanglement for Session-based Recommendation
PDF ↗Session-based Recommendation (SBR) explores dynamic user interests based on short anonymous sessions. The performance of traditional ID-collaborative SBR models is constrained by the limited session length. To this end, we focus on Multi-modal Session-based Recommendation (MMSBR), which aims to leverage modality knowledge to promote short-term user interest modeling. Previous studies cannot solve the MMSBR problem well due to issues as: (I1) Modal-invariant and specific dependencies are difficult to disentangle and unify. (I2) Multiple modal transitions produce mutual effects. (I3) Modal-aware personalized preference discrepancy exists. Thus, we propose a modality-aware diffusion-based framework MDSD with subspace disentanglement, which decouples multi-modal collaborations and provides a new paradigm that integrates modality representation generation into next-item prediction. We first employ modality-disentangled consistency graphs based on affinity semantics to extract the unified modal subspace. Then we design a cross-modal contrastive attention fusion that explores cross-modality interplay, enhancing modal alignment and consistency. To interpret personalized interest discrepancy, we realize the preference-guided multi-modal diffusion, which combines modality-aware interest generation with prediction based on collaborative signals, thus providing comprehensive recommendation results. Extensive experiments on real-world datasets demonstrate the effectiveness of MDSD.
Wind farms generate electricity from wind, offering a sustainable and eco-friendly power source. In wind farms, determining the positions of wind turbines is crucial for high energy production due to complex wake interactions. However, optimizing wind farm layouts for given wind conditions remains challenging. While exist- ing methods try to solve the problem by training a surrogate model and employing optimization algorithms based on the model, these methods require a large number of simulations to obtain highly productive layouts, which is time-consuming in large-scale tasks. Furthermore, they mostly yield less diverse layouts, making the deployment of such layouts into real-world scenarios difficult. To address these challenges, we introduce a novel conditional genera- tive modeling approach to find wind farm layouts that maximize energy production. Our method consists of four stages. First, we collect datasets that consist of layouts and their corresponding an- nual energy production (AEP). Then, we train a diffusion model conditioned on AEP and wind scenarios using the collected dataset. Specifically, we use a Graph Neural Network as the backbone for the denoising network to ensure permutation invariance. Next, we sample promising layouts from the trained diffusion model by conditioning with a high AEP and given wind scenario. We also in- troduce a local search algorithm, which enables us to adjust layouts that violate design constraints. Finally, we evaluate the generated layouts and augment the dataset. We repeat these processes itera- tively to further optimize the layouts. Throughout these processes, we effectively generate highly productive and diverse wind farm layouts in a sample-efficient manner. Experimental results demon- strate that our approach outperforms state-of-the-art methods in maximizing energy production across various scenarios.
Anomaly detection in tabular data remains challenging due to complex feature interactions and the scarcity of anomalous examples. Denoising autoencoders rely on fixed-magnitude noise, limiting adaptability to diverse data distributions. Diffusion models introduce scheduled noise and iterative denoising, but lack explicit reconstruction mappings. We propose the Diffusion-Scheduled Denoising Autoencoder (DDAE), a framework that integrates diffusion-based noise scheduling and contrastive learning into the encoding process to improve anomaly detection. We evaluated DDAE on 57 datasets from ADBench. Our method outperforms in semi-supervised settings and achieves competitive results in unsupervised settings, improving PR-AUC by up to 65% (9%) and ROC-AUC by 16% (6%) over state-of-the-art autoencoder (diffusion) model baselines. We observed that higher noise levels benefit unsupervised training, while lower noise with linear scheduling is optimal in semi-supervised settings. These findings underscore the importance of principled noise strategies in tabular anomaly detection.