论文检索

Deep Learning · Generative Models and Autoencoders

Naoki Murata, Chieh-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka, Bac Nguyen, Stefano Ermon, Yuki Mitsufuji

Recent literature has effectively leveraged diffusion models trained on continuous variables as priors for solving inverse problems. Notably, discrete diffusion models with discrete latent codes have shown strong performance, particularly in modalities suited for discrete compressed representations, such as image and motion generation. However, their discrete and non-differentiable nature has limited their application to inverse problems formulated in continuous spaces. This paper presents a novel method for addressing linear inverse problems by leveraging generative models based on discrete diffusion as priors. We overcome these limitations by approximating the true posterior distribution with a variational distribution constructed from categorical distributions and continuous relaxation techniques. Furthermore, we employ a star-shaped noise process to mitigate the drawbacks of traditional discrete diffusion models with absorbing states, demonstrating that our method performs comparably to continuous diffusion techniques with lower GPU memory consumption.

Deep Learning · Generative Models and Autoencoders

Ziseok Lee, Minyeong Hwang, Sanghyun Jo, Wooyeol Lee, Jihyung Ko, Young Bin Park, Jae-Mun Choi, Eunho Yang, Kyungsu Kim

Inference-time steering adapts pretrained diffusion and flow models to new tasks without retraining, often utilizing ratio-of-densities constructions that reweight time-indexed marginals with fixed exponents. We identify Marginal Path Collapse, a failure mode in which the intermediate density defined by such compositions becomes non-normalizable despite valid endpoints. This collapse can arise when composing heterogeneous experts trained with mismatched noise schedules (and/or negative exponents / partial supports). To address this, we provide (i) a necessary-and-sufficient Path Existence Criterion that characterizes when the composed intermediate densities are mathematically well-defined, and (ii) Adaptive path Correction with Exponents (ACE), which generalizes Feynman–Kac steering to support time-varying exponents. Our analysis reveals that ACE controls the quantile radius of the intermediate distributions, providing a theoretical mechanism for path stabilization observed in experiments. On flexible-pose scaffold decoration, a drug design task composed of de-novo, conformer, and protein-conditioned experts, ACE prevents collapse and significantly outperforms constant-exponent baselines. Furthermore, ACE improves attribute success rates in compositional image generation, establishing it as a general framework for compositional sampling.

Deep Learning · Generative Models and Autoencoders

Yuhan Wang, Suzhi Bi, Angela Yingjun Zhang

The rate-distortion-perception (RDP) tradeoff characterizes the fundamental limits of lossy compression by jointly considering bitrate, reconstruction fidelity, and perceptual quality. While recent neural compression methods have improved perceptual performance, they typically operate at fixed points on the RDP surface, requiring retraining to target different tradeoffs. In this work, we propose a training-free framework that leverages pre-trained diffusion models to traverse the entire RDP surface. Our approach integrates a reverse channel coding (RCC) module with a novel score-scaled probability flow ODE decoder. We theoretically prove that the proposed diffusion decoder is optimal for the distortion-perception tradeoff under AWGN observations and that the overall framework with the RCC module achieves the optimal RDP function in the Gaussian case. Empirical results across multiple datasets demonstrate the framework's flexibility and effectiveness in navigating the ternary RDP tradeoff using pre-trained diffusion models. Our results establish a practical and theoretically grounded approach to adaptive, perception-aware compression.

Deep Learning · Generative Models and Autoencoders

Naoki Murata, Yuhta Takida, Chieh-Hsin Lai, Toshimitsu Uesaka, Bac Nguyen, Stefano Ermon, Yuki Mitsufuji

Training-data attribution for vision generative models aims to identify which training data influenced a given output. While most methods score individual examples, practitioners often need group-level answers (e.g., artistic styles or object classes). Group-wise attribution is counterfactual: how would a model's behavior on a generated sample change if a group were absent from training? A natural realization of this counterfactual is Leave-One-Group-Out (LOGO) retraining, which retrains the model with each group removed; however, it becomes computationally prohibitive as the number of groups grows. We propose GUDA (Group Unlearning-based Data Attribution) for diffusion models, which approximates each counterfactual model by applying machine unlearning to a shared full-data model instead of training from scratch. GUDA quantifies group influence using differences in a likelihood-based scoring rule (ELBO) between the full model and each unlearned counterfactual. Experiments on CIFAR-10 and artistic style attribution with Stable Diffusion show that GUDA identifies primary contributing groups more reliably than semantic similarity, gradient-based attribution, and instance-level unlearning approaches, while achieving $\sim$100$\times$ speedup on CIFAR-10 over LOGO retraining.

Applications · Computer Vision

Zhenbang zhang, Zihui Cui, Haythem El-Messiry, Renmin Han, Zhiqiang Xu

Image-to-video (I2V) diffusion models have recently made generative inbetweening a practical reality by synthesizing semantically plausible intermediate frames between two keyframes. Among them, inference-time sampling schemes that re-use large pre-trained I2V backbones without any additional training are especially attractive. Yet current methods frequently exhibit temporal inconsistency and artifacts such as ghosting or reverse motion. A key reason is that the two trajectories are driven by distinct motion priors, each inherited from its own conditioning frame, and are simply stitched together without explicitly reconciling these priors. We introduce Motion-Residual Conflict-Aware Time Reversal (MR‑CATR), an inference-time sampling framework that aligns conflicting motion priors instead of discarding one of them or collapsing to a single start-conditioned prior. MR‑CATR first derives a motion-residual–based direction from the forward path, combined with an end-conditioned residual to form a consensus motion axis. This design suppresses bidirectional motion conflicts while still allowing end-frame information to refine the trajectory and enforce endpoint consistency. MR‑CATR can be seamlessly integrated into existing time-reversal samplers without changing model parameters. Experiments on generative inbetweening benchmarks show that our method produces videos with smoother motion, fewer artifacts, and consistently better quantitative scores and user preferences than prior strategies.

Deep Learning · Attention Mechanisms

Zhuokun Chen, Jianfei Cai, Bohan Zhuang

Generating long-form content, such as minute-long videos and extended texts, is increasingly important for modern generative models. Block diffusion improves inference efficiency via KV caching and block-wise causal inference and has been widely adopted in diffusion language models and video generation. However, in long-context settings, block diffusion still incurs substantial overhead from repeatedly computing attention over an ever-growing KV cache. We identify an underexplored property of block diffusion: cross-step redundancy of attention within a block. Our analysis shows that attention outputs from tokens outside the current block remain largely stable across diffusion steps, while block-internal attention varies significantly. Based on this observation, we propose FlashBlock, a cached block-external attention mechanism that reuses stable attention output, substantially reducing attention computation and KV cache access without modifying the diffusion process. Moreover, FlashBlock is orthogonal to sparse attention and can be combined as a complementary residual reuse strategy. When integrated, it substantially improves model accuracy under aggressive sparsification by offsetting much of the performance loss induced by sparsity. Experiments on diffusion language models and video generation demonstrate up to 1.44$\times$ higher token throughput and up to 1.6$\times$ reduction in attention time, with negligible impact on generation quality.

Social Aspects · Accountability, Transparency, and Interpretability

Zongxin Liu, Xiaoyong Xue, Sun Weidi, Shengchao Qin, Lijun Zhang

Precise, training-free editing of text-to-image diffusion models requires balancing alignment (faithful attribute manifestation), consistency (preserving non-target content), and quality (artifact-free textures). Sparse autoencoder (SAE) steering offers interpretable, smooth ``slider-like'' control by manipulating SAE feature activations derived from the text encoder; however, existing approaches rely on heuristic feature selection and manual tuning of the steering strength, leading to suboptimal trade-offs among the three objectives. We propose Sparse Relaxed-Lasso Steering (SRLS), which casts steering-vector discovery as a convex sparse recovery problem. Exploiting the affine structure of the SAE decoder, SRLS automatically identifies sparse, generalizable support sets via a Lasso objective, and then debiases the coefficients using support-restricted ridge regression. We further select the optimal steering strength using Bayesian optimization. Experiments across diverse attributes and subjects show that SRLS generally improves over other methods, yielding a better balance among alignment, consistency, and quality.

Applications · Computer Vision

Zun Wang, Jaemin Cho, Jialu Li, Han Lin, Jaehong Yoon, Yue Zhang, Mohit Bansal

Recent approaches for video generation with camera control often create anchor videos (i.e., rendered videos that approximate desired camera motions) to guide diffusion models as a structured prior, by rendering from estimated point clouds following camera trajectories. However, errors in point cloud and camera trajectory estimation often lead to inaccurate anchor videos with higher training cost and low efficiency, as the model is forced to compensate for rendering misalignments. To address these limitations, we introduce EPiC, an efficient and precise camera control learning framework that constructs well-aligned training anchor videos without the need for camera pose or point cloud estimation. Concretely, we create highly precise anchor videos by masking source videos based on first-frame visibility, which ensures strong alignment, eliminates the need for camera/point cloud estimation, and thus can be readily applied to any in-the-wild video. Furthermore, we introduce Anchor-ControlNet, a lightweight module that integrates anchor video guidance in visible regions to pretrained video diffusion models, with less than 1\% of additional parameters. EPiC achieves efficient training with substantially fewer parameters, training steps, and less data, and generalizes robustly to anchor videos made with point clouds at test time, enabling precise 3D-informed camera control. EPiC achieves SoTA performance on RealEstate10K and MiraData for I2V camera control task. Notably, EPiC also exhibits strong zero-shot generalization to video-to-video (V2V) scenarios.

Reinforcement Learning · Everything Else

Xiaoyuan Cheng, Wenxuan Yuan, Boyang Li, Yuanchao Xu, Yiming Yang, Hao Liang, Bei Peng, Robert Loftin, Zhuo Sun, Yukun Hu

Diffusion policy sampling enables reinforcement learning (RL) to represent multimodal action distributions beyond suboptimal unimodal Gaussian policies. However, existing diffusion-based RL methods primarily focus on offline setting for reward maximization, with limited consideration of safety in online settings. To address this gap, we propose Augmented Lagrangian-Guided Diffusion (ALGD), a novel algorithm for off-policy safe RL. By revisiting optimization theory and energy-based modeling, we show that the instability of primal–dual methods arises from the non-convex Lagrangian landscape. In diffusion-based safe RL, the Lagrangian can be interpreted as an energy function guiding the denoising dynamics; counter-intuitively, direct usage destabilizes both policy generation and training. ALGD resolves this issue by introducing an augmented Lagrangian that locally convexifies the energy landscape, yielding a stabilized policy generation and training, without altering the distribution of optimal policy. Theoretical analysis and extensive experiments demonstrate that ALGD is both theoretically grounded and empirically effective, achieving strong and stable performance across diverse environments.

Applications · Chemistry, Physics, and Earth Sciences

Yi He, Zimo Zhao, Yiming Yang, Xiaoyuan Cheng, Chao He, Yukun Hu

Recovering multiple physical parameters from high-dimensional optical measurements remains challenging in computational optics. We present *MMPD-Bench*, a pioneering benchmark that reframes multi-polarimetric modalities decomposition from Mueller matrix observations as a *modality fission* problem under the multi-modal learning paradigm. By replacing iterative numerical inversion with deep surrogate models, MMPD-Bench provides data, standardized solutions and evaluations to address the multi-physics modalities generation challenge. We benchmark representative architectures to this problem, including state-space models, vision transformers, conditional diffusion models, and neural operators, under a multi-faceted evaluation protocol that jointly assesses perceptual fidelity, physical consistency, robustness, and computational efficiency. Our analysis reveals non-trivial trade-offs between accuracy and robustness in accelerated high-fidelity polarimetric decomposition, highlighting key limitations of existing surrogates. To support reproducible research, we open-source the full codebase, together with a large-scale dataset of 21,412 high-resolution Mueller matrix observations acquired through extensive polarimetric measurements. We invite the community to further advance the intersection of polarization optics and multimodal representation learning.

Probabilistic Methods · Bayesian Models and Methods

Nicolas M Zilberstein, Florentin Guth, Santiago Segarra, Eero Simoncelli

Generative diffusion models can provide powerful priors for inverse problems in imaging, but existing implementations suffer from two key limitations: $(i)$ they learn only an implicit approximation of the prior density, and $(ii)$ they rely on crude likelihood approximations that introduce biases in the sampling. We address these challenges by introducing a new energy-based model trained using denoising score matching with a covariance-based regularization that enforces consistency across different inverse problems. Our approach learns explicit, normalized posterior densities for diverse linear inverse problems using a single model, while preserving the sampling capabilities of diffusion models. This enables new capabilities unavailable to score-based diffusion models: energy-guided adaptive sampling that adjusts schedules on-the-fly, unbiased MCMC correction with Metropolis-Hastings acceptance, and blind degradation estimation via Bayes rule. We validate our method on multiple datasets (MNIST, CelebA, ImageNet) and tasks (inpainting, deblurring), demonstrating competitive or superior performance to established baselines.

Deep Learning · Graph Neural Networks

Lucas Gantes, Abele Mălan, Roberto Gheda, Robert Birke, Lydia Y. Chen

Hypergraphs are graph generalizations with key applications in domains such as healthcare, where strict data privacy requirements apply, or bioinformatics, where testing new compounds is costly. However, due to their combinatorial nature, hypergraph representations are often either intractable, or introduce major information loss. For this reason, research into hypergraph synthesis is limited, and state-of-the-art approaches yield poor generation quality in terms of overall structural patterns and graph-level validity. To address such shortcomings, we introduce SuperHype, an exact and tractable hypergraph diffusion model. The core of SuperHype is the projection of graph-superposition, a novel representation that embeds a hypergraph into a multilayer graph enabling a tractable representation with no loss of generalization. To generate new samples from such representations, we introduce a Graph-Superposition Transformer that treats the superposition as an interconnected sequence of layers. Moreover, we enhance the model’s performance with hypergraph specific auxiliary features and triplet aggregation of indirect node interactions. Our evaluation on five datasets shows that SuperHype generally reproduces local and global connectivity patterns with superior fidelity than state-of-the-art baselines.

Deep Learning · Generative Models and Autoencoders

David Li, Nikita Gushchin, Dmitry Abulkhanov, Eric Moulines, Ivan Oseledets, Maxim Panov, Aleksandr Korotin

Diffusion Language Models (DLMs) have recently achieved strong results in text generation. However, their multi-step sampling leads to slow inference, limiting practical use. To address this, we extend Inverse Distillation, a technique originally developed to accelerate continuous diffusion models, to the discrete setting. Nonetheless, this extension introduces both theoretical and practical challenges. From a theoretical perspective, the inverse distillation objective lacks uniqueness guarantees, which may lead to suboptimal solutions. From a practical standpoint, backpropagation in the discrete space is non-trivial and often unstable. To overcome these challenges, we first provide a theoretical result demonstrating that our inverse formulation admits a unique solution, thereby ensuring valid optimization. We then introduce gradient-stable relaxations to support effective training. As a result, experiments on multiple DLMs show that our method, *Inverse-distilled Diffusion Language Models (IDLM)*, reduces the number of inference steps by $4 \times$-$64 \times$, while preserving the teacher model’s entropy and generative perplexity.

Deep Learning · Large Language Models

Subham Sekhar Sahoo, Jean-Marie Lemercier, Zhihan Yang, Justin Deschenaux, Jingyu Liu, John Thickstun, Ante Jukić

Diffusion language models are a promising alternative to autoregressive models due to their potential for faster generation. Among discrete diffusion approaches, Masked diffusion currently dominates, largely driven by strong perplexity on language modeling benchmarks. In this work, we present the first scaling law study of Uniform-state and interpolating discrete diffusion methods. We also show that masked diffusion models can be made approximately 12% more FLOPs-efficient when trained with a simple cross-entropy objective. We find that perplexity is informative within a diffusion family but can be misleading across families, where models with worse likelihood scaling may be preferable due to faster and more practical sampling, as reflected by the speed-quality Pareto frontier. These results challenge the view that masked diffusion is categorically the future of diffusion language modeling and that perplexity alone suffices for cross-algorithm comparison. Scaling all methods to 1.7B parameters, we show that uniform-state diffusion remains competitive on likelihood-based benchmarks and outperforms autoregressive and masked diffusion models on GSM8K, despite worse validation perplexity. Overall, **our results challenge the view that masked diffusion is categorically the future of diffusion language modeling** and that perplexity alone suffices for cross-algorithm comparison.

Deep Learning · Generative Models and Autoencoders

Qijie Zhu, Zeqi Ye, Han Liu, Zhaoran Wang, Minshuo Chen

Adaptation methods have been a workhorse for unlocking the transformative power of pre-trained diffusion models in diverse applications. Existing approaches often abstract adaptation objectives as a reward function and steer diffusion models to generate high-reward samples. However, these approaches can incur high computational overhead due to additional training, or rely on stringent assumptions on the reward such as differentiability. Moreover, despite their empirical success, theoretical justification and guarantees are seldom established. In this paper, we propose $\texttt{DOIT}$ ($\textbf{D}$oob-$\textbf{O}$riented $\textbf{I}$nference-time $\textbf{T}$ransformation), a training-free and computationally efficient adaptation method that applies to generic, non-differentiable rewards. The key framework underlying our method is a measure transport formulation that seeks to transport the pre-trained generative distribution to a high-reward target distribution. We leverage Doob's $h$-transform to realize this transport, which induces a dynamic correction to the diffusion sampling process and enables efficient simulation-based computation without modifying the pre-trained model. Theoretically, we establish a high probability convergence guarantee to the target high-reward distribution via characterizing the approximation error in the dynamic Doob's correction. Empirically, on D4RL offline RL benchmarks, our method consistently outperforms state-of-the-art baselines while preserving sampling efficiency.

Applications · Chemistry, Physics, and Earth Sciences

Zishuo Lan, Junjie Li, Lei Wang, Jincheng Wang

Autoregressive learning of time-stepping operators offers an effective approach to data-driven PDE simulation on grids. For conservation laws, however, long-horizon rollouts are often destabilized when learned updates violate global conservation and, in many applications, additional state bounds—such as nonnegative mass and densities or concentrations constrained to [0,1]. Enforcing these coupled constraints via direct next-state regression remains difficult. We introduce a framework for learning conservative transport operators on regular grids, inspired by lattice Boltzmann–style discrete-velocity transport representations. Instead of predicting the next state, the model outputs local transport operators that update cells through neighborhood exchanges, guaranteeing discrete conservation by construction. For bounded quantities, we parameterize transport within a capacity-constrained feasible set, enforcing bounds structurally rather than by post-hoc clipping. We validate FluxNet on convection-diffusion, shallow water equations, traffic flow, and spinodal decomposition. Experiments on shallow-water equations and traffic flow show improved rollout stability and physical consistency over strong baselines. On phase-field spinodal decomposition, the method enables large time-steps with long-range transport, accelerating simulation while preserving microstructure evolution in both pointwise and statistical measures.

Theory · Reinforcement Learning and Planning

Kenta Hoshino, Yashaswi Shashank Aluru, Xiyu Deng, Yorie Nakahira

This paper studies the theoretical foundations of guidance mechanisms in continuous-time score-based diffusion models. We adopt Doob’s h-transform as a principled framework for characterizing ideal guided diffusion processes and analyze the discrepancy between ideal and approximate guidance. Our analysis provides explicit error bounds and yields probabilistic guarantees on satisfying prescribed constraints, which are particularly important for safety-critical planning. We further show that the Doob-based formulation induces a stochastic optimal control problem, enabling practical guidance design without additional model training. We demonstrate the effectiveness of the proposed framework on robotic navigation tasks, including language-conditioned planning.

Deep Learning · Generative Models and Autoencoders

Xiao Li, Yixuan Jia, Zekai Zhang, Xiang Li, Lianghe Shi, Jinxin Zhou, Zhihui Zhu, Liyue Shen, Qing Qu

Diffusion models are effective generative frameworks with strong representation learning capabilities, yet the intrinsic properties that govern their semantic structure and generalization remain poorly understood. Drawing inspiration from self-supervised representation learning (SSL), we introduce an evaluation framework that decomposes diffusion features into a perturbation invariant component and a residual component induced by noise and augmentations. From this decomposition we derive the Invariant Contamination Ratio (ICR), a Fisher-based metric that measures how residual, augmentation-sensitive energy contaminates invariant signal in feature space. We use this framework to analyze both discriminative and generative behavior. On the representation side, we find invariance peaks at intermediate noise levels, which also yield the best downstream classification performance. On the generative side, we study how training transitions from genuine generalization to memorization in data-limited regimes, and find that $\mathrm{ICR}$ serves as a sensitive training time indicator of the early learning phenomenon: rising residual energy along Fisher directions marks the onset of memorization, detectable from training features alone without external evaluators or held-out test sets. Overall, our results show diffusion models can be monitored from a self-supervised perspective via the geometry of their learned representations.

Applications · Computer Vision

Carmine Zaccagnino, Fabio Quattrini, Enis Simsar, Marta Gazulla, Rita Cucchiara, Alessio Tonioni, Silvia Cascianelli

Flow matching models have recently emerged as an efficient alternative to diffusion, especially for text-guided image generation and editing, offering faster inference through continuous-time dynamics. However, existing flow-based editors predominantly support global or single-instruction edits and struggle with multi-instance scenarios, where multiple parts of a reference input must be edited independently without semantic interference. We identify this limitation as a consequence of globally conditioned velocity fields and joint attention mechanisms, which entangle concurrent edits. To address this issue, we introduce Instance-Disentangled Attention, a mechanism that partitions joint attention operations, enforcing binding between instance-specific textual instructions and spatial regions during velocity field estimation. We evaluate our approach on both natural image editing and a newly introduced benchmark of text-dense infographics with region-level editing instructions. Experimental results demonstrate that our approach promotes edit disentanglement and locality while preserving global output coherence, enabling single-pass, instance-level editing.

Deep Learning · Large Language Models

Mingyu Cao, Alvaro Correia, Christos Louizos, Shiwei Liu, Lu Yin

Diffusion Language Models (DLMs) generate text by iteratively denoising a masked sequence, repeatedly deciding which positions to commit at each step. Standard decoding follows a greedy rule, unmasking the most confident positions, yet this local choice can lock the model into a suboptimal unmasking order, especially on reasoning-heavy prompts. We present Search Or AcceleRate (SOAR), a training-free decoding algorithm that adapts its behavior to the model’s uncertainty. When confidence is low, SOAR briefly widens the search over alternative unmasking decisions to avoid premature commitments; when confidence is high, it collapses the search and decodes many positions in parallel to reduce the number of denoising iterations. Across mathematical reasoning and code generation benchmarks (GSM8K, MBPP, HumanEval) on Dream-7B and LLaDA-8B, SOAR improves generation quality while maintaining competitive inference speed, offering a practical way to balance quality and efficiency in DLM decoding.