The integration of single-cell RNA sequencing (scRNA-seq) and spatial transcriptomics (ST) data is crucial for understanding gene expression in spatial context. Existing methods for such integration have limited performance, with structural similarity often below 60%, We attribute this limitation to the failure to consider causal relationships between genes. We present CausalGeD, which combines diffusion and autoregressive processes to leverage these relationships. By generalizing the Causal Attention Transformer from image generation to gene expression data, our model captures regulatory mechanisms without predefined relationships. Across 10 tissue datasets, CausalGeD outperforms state-of-the-art baselines by 5- 32% in key metrics, including Pearson's correlation and structural similarity, advancing both technical and biological insights.
论文检索
输入标题、作者或关键词，从 9,256 篇学术成果中精准定位
Knowledge tracing (KT), which is instrumental in monitoring and forecasting students' knowledge states throughout their learning trajectory in online learning environments, has over the past decade garnered widespread attention due to its pivotal role in facilitating personalized education. Existing KT approaches were mainly invented from the model-centric perspective to overcome the sequence modeling difficulty while not exploiting the potential information of sparsity, thereby limiting their performance. To make full use of the information in the dataset, this paper proposes a data-centric knowledge tracing paradigm, termed LPDG, aiming to generate interaction data between students and exercises by revealing students' Learning Patterns and facilitating the Generation of ideal training Data. Specifically, we propose a learning patterns-guided exercise sequence regenerator, which incorporates Transformer and a tailor-made pattern enhancer, thereby aiding in the extraction of valuable information for generating high-quality training data. Moreover, we devise a learning patterns-guided pseudo-label generator, which leverages the diffusion process to construct pseudo-labels for the regenerated sequences. Afterwards, the fully generated ideal data is incorporated into the training data, and we integrate this framework with various model-centric approaches in KT. Finally, experimental results across datasets clearly demonstrate the efficacy of our proposed LPDG framework.
This paper studies grid-free point process modeling under varying fluid parameters. Existing methods rely on grid-based approaches or fixed parameters, making it challenging to handle complex nonlinear dynamics and out-of-distribution (OOD) scenarios. To address this, we propose Adaptive Perturbation Graph ODE (AGODE), a novel framework that integrates three key innovations: (1) an adaptive conditioning mechanism for physical parameter adaptation(2) a continuous graph neural ODE for spatiotemporal evolution modeling, and (3) a perturbation module with mutual information maximization for uncertainty quantification. AGODE employs graph neural networks to encode unstructured point cloud data into latent dynamics governed by neural ODEs, where physical parameters are injected through context-aware conditioning vectors. The perturbation module generates diverse trajectory samples by introducing stochastic noise during ODE integration, while contrastive learning aligns predictions with physical contexts to filter implausible outcomes. Extensive experiments across five fluid dynamics benchmarks (Prometheus, Navier-Stokes, Spherical-SWE, 3D Reaction-Diffusion, ERA5) demonstrate AGODE's state-of-the-art performance. Specifically, AGODE achieves MSE of 0.0302/0.0312 (in-domain/OOD) on Prometheus, outperforming PURE by 6.5%/5.0%, and reduces Navier-Stokes errors by 28.1% compared to physics-informed NMO. Notably, AGODE maintains superior OOD generalization with only 2.9% average error increase versus 7.8% for baselines, while its uncertainty quantification improves prediction reliability by 41% (95% confidence interval coverage). These results validate AGODE's capabilities in continuous spatiotemporal modeling, multi-parameter adaptation, and robust uncertainty estimation for complex fluid systems.
Recent successes in diffusion probabilistic models have demonstrated their strength in modeling and generating different types of data, paving the way for their application in generative time series forecasting. However, most existing diffusion based approaches rely on sequential models and unimodal latent variables to capture global dependencies and model entire observable data, resulting in difficulties when it comes to highly stochastic time series data. In this paper, we propose a novel Stochastic Diffusion (StochDiff) model that integrates the diffusion process into time series modeling stage and utilizes the representational power of the stochastic latent spaces to capture the variability of the stochastic time series data. Specifically, the model applies diffusion module at each time step within the sequential framework and learns a step-wise, data-driven prior for generative diffusion process. These features enable the model to effectively capture complex temporal dynamics and the multi-modal nature of the highly stochastic time series data. Through extensive experiments on real-world datasets, we demonstrate the effectiveness of our proposed model for probabilistic time series forecasting, particularly in scenarios with high stochasticity. Additionally, with a real-world surgical use case, we highlight the model's potential in a medical application.
Fine-grained traffic management and prediction are fundamental to key applications such as autonomous driving, lane change guidance, and traffic signal control. However, obtaining lane-level traffic data has become a critical bottleneck for data-driven models due to limitations in the types and number of sensors and issues with the accuracy of tracking algorithms. To address this, we propose the Fine-grained Road Traffic Inference (FRTI) task, which aims to generate more detailed lane-level traffic information using limited road data, providing a more energy-efficient and cost-effective solution for precise traffic management. This task is abstracted as the first scene of the spatio-temporal graph node generation problem. We designed a two-stage framework-RoadDiff-to solve the FRTI task. This framework leverages the Road-Lane Correlation Autoencoder-Decoder and the Lane Diffusion Module to fully utilize the limited spatio-temporal dependencies and distribution relationships of road data to accurately infer fine-grained lane traffic states. Based on existing research, we designed several baseline models with the potential to solve the FRTI task and conducted extensive experiments on six datasets representing different road conditions to validate the effectiveness of the RoadDiff model in addressing the FRTI task. The relevant datasets and code are available at https://github.com/ShuhaoLii/RoadDiff.
ESCFD: Probabilistic Flow Diffusion Model for Accelerated High-Quality Single-Cell RNA-seq Data Synthesis
PDF ↗Single-cell RNA sequencing (scRNA-seq) technology has emerged as a powerful tool for studying complex biological systems and diseases with high resolution. However, generating sufficient high quality scRNA-seq samples for accurate downstream analyses and ensuring reproducibility remains a challenge. Previous methods often produce low-quality samples or a limited number of useful specific cell subpopulations. To address this issue, we propose a novel Efficient Single-Cell RNA-seq Flow Diffusion (named ESCFD) method based on the Diffusion Model, capable of synthesizing large-scale, high-quality scRNA-seq samples and generating either ''holistic'' or specific cellular subpopulations within a single framework. A pre-guidance mechanism is designed for synthesizing specific cellular subpopulations of high quality, and a post-guidance mechanism was designed to accelerate the sampling of scRNA-seq samples. The ESCFD can efficiently synthesize large-scale and high-quality scRNA-seq samples for various downstream tasks, surpassing previous work with faster sampling speed. Experimental results demonstrate that the ESCFD model achieves state-of-the-art performance in cell classification and cell distribution similarity when evaluated on three scRNA-seq benchmarks, and verified the effectiveness of accelerated sampling. Visualization experiments also reveal the ESCFD's ability to synthesize specific cellular subpopulations. This innovative approach has the potential to significantly advance our understanding of complex biological systems and accelerate the development of novel therapeutic interventions and personalized medicine strategies.
Predicting the dynamics of complex systems is crucial for various scientific and engineering applications. The accuracy of predictions depends on the model's ability to capture the intrinsic dynamics. While existing methods capture key dynamics by encoding a low-dimensional latent space, they overlook the inherent multiscale structure of complex systems, making it difficult to accurately predict complex spatiotemporal evolution. Therefore, we propose a Multiscale Diffusion Prediction Network (MDPNet) that leverages the multiscale structure of complex systems to discover the latent space of intrinsic dynamics. First, we encode multiscale features through a multiscale diffusion autoencoder to guide the diffusion model for reliable reconstruction. Then, we introduce an attention-based graph neural ordinary differential equation to model the co-evolution across different scales. Extensive evaluations on representative systems demonstrate that the proposed method achieves an average prediction error reduction of 53.23% compared to baselines, while also exhibiting superior robustness and generalization.
Diffusion bridges have shown potential in paired image-to-image (I2I) translation tasks. However, existing methods are limited by their unidirectional nature, requiring separate models for forward and reverse translations. This not only doubles the computational cost but also restricts their practicality. In this work, we introduce the Bidirectional Diffusion Bridge Model (BDBM), a scalable approach that facilitates bidirectional translation between two coupled distributions using a single network. BDBM leverages the Chapman-Kolmogorov Equation for bridges, enabling it to model data distribution shifts across timesteps in both forward and backward directions by exploiting the interchangeability of the initial and target timesteps within this framework. Notably, when the marginal distribution given endpoints is Gaussian, BDBM's transition kernels in both directions possess analytical forms, allowing for efficient learning with a single network. We demonstrate the connection between BDBM and existing bridge methods, such as Doob's h-transform and variational approaches, and highlight its advantages. Extensive experiments on high-resolution I2I translation tasks demonstrate that BDBM not only enables bidirectional translation with minimal additional cost but also outperforms state-of-the-art bridge models. Our source code is available at https://github.com/kvmduc/BDBM.
Classical time-series forecasting methods rely on learning from a fixed context window of past observations and extrapolating to a fixed number of future steps. They are therefore limited in flexibility and effectiveness in modeling tasks of varying historical lengths or future horizons, requiring re-training for each setting. We propose Contextual Meta-Adaptation (CMA), a unified scheme that captures history contexts and extrapolates future horizons with a single training. CMA integrates denoising diffusion capability which can sequentially generate a future series with maximum likelihood and remains consistent with observed history and past generations. We develop an efficient update strategy using gradient-based meta-learning for full-parameter tuning during training and LoRA adaptation at testing time. We validate our CMA scheme on in-context learning with longer observed history, ex(tend)-context learning with up to 3 times longer future horizons, and transfer-learning across different source-target domains. Our CMA outperforms existing methods by an average of 7% on six academic datasets and a significant 16% improvement on four realistic marketing datasets.
Graph Neural Networks (GNNs) have demonstrated strong representation learning capabilities for graph-based tasks. Recent advances on GNNs leverage geometric properties, such as curvature, to enhance their representation capabilities by modeling complex connectivity patterns and information flow within graphs. However, most existing approaches primarily focus on discrete graph topology, overlooking diffusion dynamics and task-specific dependencies essential for effective learning. To address this, we propose a learnable integration of Bakry-Émery curvature, which captures both structural and diffusion aspects of information propagation. We develop an efficient, learnable approximation strategy, making curvature computation scalable for large graphs. Furthermore, we introduce an adaptive depth mechanism that dynamically adjusts message-passing layers per vertex based on its curvature, ensuring efficient propagation. Our theoretical analysis establishes a link between curvature and feature distinctiveness, showing that high-curvature vertices require fewer layers, while low-curvature ones benefit from deeper propagation. Extensive experiments on diverse downstream tasks validate the effectiveness of our approach, showing that the proposed depth-adaptive mechanism consistently uplifts the performance of a wide range of GNN architectures.
Probabilistic time series imputation has been widely applied in real-world scenarios due to its ability for uncertainty estimation and denoising diffusion probabilistic models (DDPMs) have achieved great success in probabilistic time series imputation tasks with its power to model complex distributions. However, current DDPM-based probabilistic time series imputation methodologies are confronted with two types of challenges: 1) The backbone modules of the denoising parts are not capable of achieving sequence modeling with low time complexity. 2) The architecture of denoising modules can not handle the dependencies in the time series data effectively. To address the first challenge, we explore the potential of state space model, namely Mamba, as the backbone denoising module for DDPMs. To tackle the second challenge, we carefully devise several SSM-based blocks for time series data modeling. Experimental results demonstrate that our approach can achieve state-of-the-art time series imputation results on multiple real-world datasets. Our datasets and code are available at https://github.com/decisionintelligence/SSD-TS/
Information diffusion prediction is fundamental for forecasting user participation in information sharing on social networks, such as retweets on Twitter. Existing methods typically extract user relationships from social networks and historical interactions, while further capturing contextual information within the specific diffusion process. However, these methods have several limitations: (1) They often utilize sequential diffusion process for prediction and simplify differentiated influences among participants; (2) They capture user relationships on the entire graph for all users, in which most information is not necessary for a specific diffusion process and is too inefficient for real-world large-scale networks. To tackle these limitations, we propose a novel and scalable model SILN, for sphere-based information diffusion prediction on large social networks. Specifically, SILN features three components. First, we integrate two kinds of sphere effects in terms of structural and temporal views, which learn an enhanced cascade representation. Second, SILN designs an efficient learning scheme based on the cascade-specific subgraph, which significantly reduces the entire graph computation to smaller subgraphs. Third, to facilitate subgraph extraction, we develop an optimized graph storage technique to allow constant-time neighbor access and reduce the storage cost by about 30% in practice. Extensive experiments on six real-world datasets validate that SILN consistently outperforms seven state-of-the-art competitors in prediction performance while exhibiting exceptional time and space efficiency on million-node social networks.
Microservices architecture has become increasingly popular in modern software systems, yet its complexity also poses challenges for failure diagnosis. Existing literature has deficiencies in multimodal data mining, mainly in two aspects: first, the mining of multimodal data is insufficient, failing to fully exploit the rich information contained in different types of data; second, the analysis of spatiotemporal features of multimodal data is not thorough, failing to fully explore the potential associations of data in the temporal and spatial dimensions. To address these issues, this paper proposes a new method for anomaly detection in microservices. We transform three types of modal data into the wavelet domain and achieve fusion among modalities based on an attention mechanism, fully utilizing the inherent characteristics of multimodal data in the frequency and temporal domains. This fusion process can extract new information that does not exist in the original modalities, thereby enhancing the model's ability to detect anomalies. In addition, we propose a new diffusion probabilistic model (DDPM) based on spatiotemporal graphs, which combines spatiotemporal learning capabilities with the uncertainty measurement of DDPM to generate future samples in a non-autoregressive manner, achieving multi-horizon prediction. Experimental results show that our method significantly improves anomaly detection performance on three public datasets and a real-world production system, demonstrating its effectiveness in microservices anomaly detection.
Synthetic Electronic Health Record (EHR) time-series generation is crucial for advancing clinical machine learning models, as it helps address data scarcity by providing more training data. However, most existing approaches focus primarily on replicating statistical distributions and temporal dependencies of real-world data. We argue that fidelity to observed data alone does not guarantee better model performance, as common patterns may dominate, limiting the representation of rare but important conditions. This highlights the need for generate synthetic samples to improve performance of specific clinical models to fulfill their target outcomes. To address this, we propose TarDiff, a novel target-oriented diffusion framework that integrates task-specific influence guidance into the synthetic data generation process. Unlike conventional approaches that mimic training data distributions, TarDiff optimizes synthetic samples by quantifying their expected contribution to improving downstream model performance through influence functions. Specifically, we measure the reduction in task-specific loss induced by synthetic samples and embed this influence gradient into the reverse diffusion process, thereby steering the generation towards utility-optimized data. Evaluated on six publicly available EHR datasets, TarDiff achieves state-of-the-art performance, outperforming existing methods by up to 20.4% in AUPRC and 18.4% in AUROC. Our results demonstrate that TarDiff not only preserves temporal fidelity but also enhances downstream model performance, offering a robust solution to data scarcity and class imbalance in healthcare analytics.
Synthetic human trajectory data becoming increasingly prominent in various applications, including urban planning, traffic control, and crowd monitoring. Recent neural generative models for human trajectory data mostly follow an unconditional generative paradigm that relies on a pure data-driven imitative learning scheme, without considering the rich context of human mobility (e.g., social events or weather conditions) which may significantly impact the underlying human mobility patterns. Against this background, we propose Marionette, a Manipulatable generative model for human trajectory data with fine-grained conditions. Specifically, Marionette integrates both global and partial mobility-related contexts and extracts both sequence-level and event-level conditions. Afterward, it designs fine-grained and cascading conditioning mechanisms for modeling the temporal and spatial dynamics based on diffusion-alike Temporal Point Processes (TPPs) and discrete diffusion models, respectively, offering fine-grained controllable generative modeling of human trajectory data with both global and partial mobility-related contexts. We conduct a thorough evaluation on two real-world human trajectory datasets against a sizeable collection of baselines. Results show that our Marionette consistently outperforms the best baselines by 13.96-54.13% on statistical and distributional similarity metrics and by 9.36-40.63% in task-based data utility evaluation. Ablation studies verify our key design choices. Case studies also demonstrate the manipulability of Marionette in generating data in previously unseen scenarios.
Unlocking the Power of Diffusion Models in Sequential Recommendation: A Simple and Effective Approach
PDF ↗In this paper, we focus on the often-overlooked issue of embedding collapse in existing diffusion-based sequential recommendation models and propose ADRec, an innovative framework designed to mitigate this problem. Diverging from previous diffusion-based methods, ADRec applies an independent noise process to each token and performs diffusion across the entire target sequence during training. ADRec captures token interdependency through auto-regression while modeling per-token distributions through token-level diffusion. This dual approach enables the model to effectively capture both sequence dynamics and item representations, overcoming the limitations of existing methods. To further mitigate embedding collapse, we propose a three-stage training strategy: (1) pre-training the embedding weights(2) aligning these weights with the ADRec backbone, and (3) fine-tuning the model. During inference, ADRec applies the denoising process only to the last token, ensuring that the meaningful patterns in historical interactions are preserved. Our comprehensive empirical evaluation across six datasets underscores the effectiveness of ADRec in enhancing both the accuracy and efficiency of diffusion-based sequential recommendation systems.
Large-scale human mobility data contains rich spatial and temporal information for urban sensing, crowd flow modeling, and urban planning. However, it is usually difficult to access wide-coverage, long-term, and consistent-time human mobility data. Most of the publicly available datasets are actually only records of discontinuous trajectories of a very small portion of urban citizens in asynchronous time due to the limited usage of apps for location data collection or the limited number of volunteers. To address this problem and empower open urban research, this paper constructs a high-quality human mobility dataset by generating large-scale citizen trajectories based on massive cellular signaling data. Particularly, we first propose a heatmap diffusion module to generate a probability heatmap that produces plausible trajectories at both the individual and city scales. Then, we propose a masked trajectory AutoEncoder, which can generate individual trajectory embeddings from partially given or empty trajectories. Third, a flexible framework is provided to incorporate the heatmap diffusion module with the masked trajectory embeddings, demonstrating significant flexibility in handling both fully masked trajectories for city-wide analysis and partially masked trajectories for specific locations. We have conducted extensive experiments to validate the utility of the regenerated trajectories at both individual and region levels for various applications. Numerous case studies further illustrate that our model learns not only the distribution of the trajectories but also the semantics of different urban areas. In summary, this paper provides a Heatmap Diffusion framework based on a Masked Trajectory AutoEncoder to regenerate flexible trajectories for open urban research. Correspondingly, we will try to open a large-scale human mobility data service for open urban research. Further information can be found at https://github.com/Rising0321/FinalOpenUR.
Telegram is a widely adopted instant messaging platform. It has become worldwide popular because of its emphasis on privacy and its social network features such as channels-virtual rooms in which only the admins can post and broadcast messages to all the subscribers. Channels are used to deliver live updates (e.g., weather alerts) and content to a large audience (e.g., COVID-19 announcements) but unfortunately also to disseminate radical ideologies and coordinate attacks such as the Capitol Hill riot. This paper introduces the TGDataset, the most extensive publicly available collection of Telegram channels, comprising 120,979 channels and over 400 million messages. We outline the data collection process and provide a comprehensive overview of the data set. Using language detection, we identify the predominant languages within the dataset. We then focus on English channels, employing topic modeling to analyze the subjects they cover. Finally, we discuss some use cases in which our dataset can be instrumental in understanding the Telegram ecosystem and studying the diffusion of questionable news. Alongside the raw dataset, we release the scripts used in our analysis, as well as a list of channels associated with a novel conspiracy theory known as Sabmyk.
In the rapidly progressing field of generative models, the development of efficient and high-fidelity text-to-image diffusion systems represents a significant frontier. This study introduces YaART, a novel production-grade text-to-image cascaded diffusion model aligned to human preferences using Reinforcement Learning from Human Feedback (RLHF). During the development of YaART, we especially focus on the choices of the model and training dataset sizes, the aspects that were not systematically investigated for text-to-image cascaded diffusion models before. In particular, we comprehensively analyze how these choices affect both the efficiency of the training process and the quality of the generated images, which are highly important in practice. Furthermore, we demonstrate that models trained on smaller datasets of higher-quality images can successfully compete with those trained on larger datasets, establishing a more efficient scenario of diffusion models training. From the quality perspective, YaART is consistently preferred by users over many existing state-of-the-art models. The proposed system is integrated in five commercial products. YaART is also distributed for B2C clients via mobile and web applications and for B2B clients via API.
Recent advances in generative artificial intelligence have revolutionized information retrieval and content generation, opening up new opportunities for the e-commerce industry. Alignment learning between small models and parallel corpora cannot meet current needs. The success of ChatGPT demonstrates that large models need to first establish a fundamental understanding, and then utilize high-quality corpora for generation. Having a large model foundation is indispensable. In this paper, we establish a fundamental 10B multimodal model foundation for multimodal generation tasks and propose a scene-based alignment learning approach called conditional sample supervised fine-tuning for downstream generation tasks. Meanwhile, diffusion models are known to be vulnerable to outliers in training data. To address this, we utilize an alternative diffusion loss function that preserves the high quality of generated data like the original squared L2 loss while being robust to outliers.In practical test sets, the multimodal foundation fully demonstrates its alignment and comprehension abilities for graphic and textual content. Additionally, conditional fine-tuning and the design of the loss function significantly enhance the quality of generated content. The quality rate of images has increased by 34.3 percentage points, and prompt control has improved by 19.8 percentage points. The application of our framework in Baidu Search Ads has led to significant revenue growth. For instance, ads with generated image creatives have achieved a 29% higher click-through rate (CTR), resulting in a daily consumption of 3 million yuan.