论文检索

Deep Learning · Generative Models and Autoencoders

Zichen Zhong, Haoliang Sun, Yukun Zhao, Yongshun Gong, Yilong Yin

Flow Matching enables simulation-free training of generative models on Riemannian manifolds, yet sampling typically still relies on numerically integrating a probability-flow ODE. We propose Riemannian MeanFlow (RMF), extending MeanFlow to manifold-valued generation where velocities lie in location-dependent tangent spaces. RMF defines an average-velocity field via parallel transport and derives a Riemannian MeanFlow identity that links average and instantaneous velocities for intrinsic supervision. We make this identity practical in a log-map tangent representation, avoiding trajectory simulation and heavy geometric computations. For stable optimization, we decompose the RMF objective into two terms and apply conflict-aware multi-task learning to mitigate gradient interference. RMF also supports conditional generation via classifier-free guidance. Experiments on spheres, tori, and SO(3) demonstrate competitive one-step sampling with improved quality–efficiency trade-offs and substantially reduced sampling cost.

Deep Learning · Generative Models and Autoencoders

Kairan Zhao, Eleni Triantafillou, Peter Triantafillou

Generative models have been shown to "memorize" certain training data, leading to verbatim or near-verbatim generating images, which may cause privacy concerns or copyright infringement. We introduce Guidance Using Attractive-Repulsive Dynamics (GUARD), a novel framework for memorization mitigation in text-to-image diffusion models. GUARD adjusts the image denoising process to guide the generation away from an original training image and towards one that is distinct from training data while remaining aligned with the prompt, guarding against reproducing training data, without hurting image generation quality. We propose a concrete instantiation of this framework, where the positive target that we steer towards is given by a novel method for (cross) attention attenuation based on (i) a novel statistical mechanism that automatically identifies the prompt positions where cross attention must be attenuated and (ii) attenuating cross-attention in these per-prompt locations. The resulting GUARD offers a surgical, dynamic per-prompt inference-time approach that, we find, is by far the most robust method in terms of consistently producing state-of-the-art results for memorization mitigation across two architectures and for both verbatim and template memorization, while also improving upon or yielding comparable results in terms of image quality.

Deep Learning · Generative Models and Autoencoders

Baoquan Zhang, Bingqi Shan, Shihao Fang, Kenghong Lin, Xutao Li, Yunming Ye

Speculative Jacobi Decoding (SJD) is an important approach for accelerating autoregressive image generation. Although SJD has shown superior performance, recent studies point out that it usually suffers from a token ambiguity issue during token verification but its reason can not be well explained. To figure out this reason, in this paper, we conduct a visualization analysis on vision token and find that different from text tokens, vision tokens generally corresponds to some local, small, and unclear vision details, which means only using single token is difficult to accurately express a certain semantic, thereby causing token ambiguity issue. To this end, we propose a novel Speculative Jacobi Decoding with Semantics Verification (called SJD-SV), for accelerating autoregressive image generation. The key idea is that leveraging the strong correction characters between tokens to recognize semantic-aware token subsequence and then instead of perform token-by-token verification, turning to perform verification on semantic-aware token subsequence level for accelerating image generation. In particular, our method is plug-in, which can be directly integrated into existing SJD and its variants. Extensive experiments on various datasets show that existing SJD methods achieve significant performance improvement after integrating our SJD-SV method.

Deep Learning · Generative Models and Autoencoders

Ying Shen, Zhiyang Xu, Jiuhai Chen, Shizhe Diao, Jiaxin Zhang, Yuguang Yao, Joy Rimchala, Ismini Lourentzou, Lifu Huang

Recent advances in multimodal foundation models unifying image understanding and generation have opened exciting avenues for tackling a wide range of vision-language tasks within a single framework. Despite progress, existing unified models typically require extensive pretraining, and many of these models suffer from slow image generation speeds, limiting their practical deployment in real-time or resource-constrained settings. In this work, we propose Layerwise Timestep-Expert Flow-based Transformer (LaTtE-Flow), a novel architecture that improves the efficiency of diffusion/flow-based transformer within the unified model setting. LaTtE-Flow builds upon powerful pretrained Vision-Language Models (VLMs) to inherit strong multimodal understanding capabilities, and extends them with a novel Layerwise Timestep Experts flow-based architecture for efficient image generation. LaTtE-Flow distributes the flow-matching process across specialized groups of Transformer layers, each responsible for a distinct subset of timesteps. This design significantly improves sampling efficiency by activating only a small subset of layers at each sampling timestep. To further enhance performance, we propose a Timestep-Conditioned Residual Attention mechanism for efficient information reuse across layers. Experiments demonstrate that LaTtE-Flow achieves strong performance on multimodal understanding tasks, while achieving competitive image generation quality with around 6× faster inference speed compared to recent unified multimodal models.

Deep Learning · Generative Models and Autoencoders

Jian-Feng Cai, Haixia Liu, Zhengyi Su, Chao Wang

Classifier-free guidance (CFG) is a widely used technique for controllable generation in diffusion and flow-based models. Despite its empirical success, CFG relies on a heuristic linear extrapolation that is often sensitive to the guidance scale. In this work, we provide a principled interpretation of CFG through the lens of optimization. We demonstrate that the velocity field in flow matching corresponds to the gradient of a sequence of smoothed distance functions, which guides latent variables toward the scaled target image set. This perspective reveals that the standard CFG formulation is an approximation of this gradient, where the prediction gap, the discrepancy between conditional and unconditional outputs, governs guidance sensitivity. Leveraging this insight, we reformulate the CFG sampling as a homotopy optimization with a manifold constraint. This formulation necessitates a manifold projection step, which we implement via an incremental gradient descent scheme during sampling. To improve computational efficiency and stability, we further enhance this iterative process with Anderson Acceleration without requiring additional model evaluations. Our proposed methods are training-free and consistently refine generation fidelity, prompt alignment, and robustness to the guidance scale. We validate their effectiveness across diverse benchmarks, demonstrating significant improvements on large-scale models such as DiT-XL-2-256, Flux, and Stable Diffusion 3.5. Code is available in the supplementary materials.

Deep Learning · Generative Models and Autoencoders

Xuanyi Liu, Deyi Ji, Liqun Liu, Lanyun Zhu, Xuhang Chen, Qianxiong Xu, Peng Shu, Huan Yu, Jie Jiang, Feng Gao 等

Sparse camera-conditioned image-to-video generation presents a pivotal challenge: synthesizing geometrically consistent 3D motion from minimal pose cues. Existing methods, which largely rely on dense supervision or naive interpolation, suffer from severe pose drift and motion discontinuities due to the lack of robust 3D priors. In this paper, we introduce \textbf{CamGeo}, a novel framework that distills rich 3D geometric knowledge from a pre-trained video-to-3D model (VGGT) directly into the diffusion backbone. To achieve this without incurring inference latency, we propose a training-only distillation strategy. Specifically, CamGeo incorporates: (1) keyframe trajectory distillation that enforces cycle-consistency with sparse input poses, (2) cross-frame consistency distillation with both camera trajectory and depth constraints to generate consistent structure across unsupervised frames, and (3) a three-stage coarse-to-fine curriculum learning, progressively scales geometric complexity, from global structure coherence to fine-grained refinement, achieving stable optimization. Extensive experiments demonstrate that CamGeo achieves consistent improvements under various sparsity ratios.

Andrew Gordon, Daniel Murfet, William Snell, Garrett Baker, George Wang, Stan van Wingerden

Spectroscopy infers the internal structure of physical systems by measuring their response to perturbations. We apply this principle to neural networks: perturbing the data distribution by upweighting a token $y$ in context $x$, we measure the model's response via susceptibilities $\chi_{xy}$, which are covariances between component-level observables and the perturbation computed over a localized Gibbs posterior via stochastic gradient Langevin dynamics (SGLD). Theoretically, we show that susceptibilities decompose as a sum over \emph{modes} of the data distribution, explaining why tokens that follow their contexts ``for similar reasons'' cluster together in susceptibility space. Empirically, we apply this methodology to Pythia-14M, developing a conductance-based clustering algorithm that identifies 510 interpretable clusters ranging from grammatical patterns to code structure to mathematical notation. Comparing to sparse autoencoders, 50\% of our clusters match SAE features, validating that both methods recover similar structure.

Deep Learning · Generative Models and Autoencoders

Jing Xu, Yuexiao Ma, Songwei Liu, Xuzhe Zheng, Shiwei Liu, Chenqian Yan, Xiawu Zheng, Rongrong Ji, Fei Chao, WANG

Autoregressive video generation paradigms offer theoretical promise for long video synthesis, yet their practical deployment is hindered by the computational burden of sequential iterative denoising. While cache reuse strategies can accelerate generation by skipping redundant denoising steps, existing methods rely on coarse-grained chunk-level skipping that fails to capture fine-grained pixel dynamics. This oversight is critical: pixels with high motion require more denoising steps to prevent error accumulation, while static pixels tolerate aggressive skipping. We formalize this insight theoretically by linking cache errors to residual instability, and propose $\textbf{MotionCache}$, a motion-aware cache framework that exploits inter-frame differences as a lightweight proxy for pixel-level motion characteristics. MotionCache employs a coarse-to-fine strategy: an initial warm-up phase establishes semantic coherence, followed by motion-weighted cache reuse that dynamically adjusts update frequencies per token. Extensive experiments on state-of-the-art models like SkyReels-V2 and MAGI-1 demonstrate that MotionCache achieves significant speedups of $\textbf{6.28}\times$ and $\textbf{1.64}\times$ respectively, while effectively preserving generation quality (VBench: 1%$\downarrow$ and 0.01%$\downarrow$ respectively). The code is available at https://anonymous.4open.science/r/MotionCache.

General Machine Learning · Causality

Ayush Khot, Miruna Oprescu, Maresa Schröder, Ai Kagawa, Xihaier Luo

Causal inference in spatial domains faces two intertwined challenges: (1) unmeasured spatial factors, such as weather, air pollution, or mobility, that confound treatment and outcome, and (2) interference from nearby treatments that violate standard no-interference assumptions. While existing methods typically address one by assuming away the other, we show they are deeply connected: *interference reveals structure* in the latent confounder. Leveraging this insight, we propose the **Spatial Deconfounder**, a two-stage method that reconstructs a substitute confounder from local treatment vectors using a conditional variational autoencoder (C-VAE) with a spatial prior, then estimates causal effects via a flexible outcome model. We show that this approach enables nonparametric identification of both direct and spillover effects under weak assumptions—without requiring multiple treatment types or a known model of the latent field. Empirically, we extend ```SpaCE```, a benchmark suite for spatial confounding, to include treatment interference, and show that the Spatial Deconfounder consistently improves effect estimation across real-world datasets in environmental health and social science. By turning interference into a multi-cause signal, our framework bridges spatial and deconfounding literatures to advance robust causal inference in structured spatial data.

Deep Learning · Generative Models and Autoencoders

Dhruvesh Patel, Benjamin Rozonoyer, Gaurav Pandey, Tahira Naseem, Ramón Astudillo, Andrew McCallum

In many domains generating variable length sequences through insertions provides greater flexibility over autoregressive models. However, the action space of insertion models is much larger than that of autoregressive models (ARMs) making the learning challenging. To address this, we incorporate trainable order dynamics into the target rates for discrete flow matching, and show that with suitable choices of parameterizations, joint training of the target order dynamics and the generator is tractable without the need for numerical simulation. As the generative insertion model, we use a variable length masked diffusion model, which generates by inserting and filling mask tokens. On graph traversal tasks for which a locally optimal insertion order is known, we explore the choices of parameterization empirically and demonstrate the trade-offs between flexibility, training stability and generation quality. On de novo small molecule generation, we find that the learned order dynamics leads to a significant increase in validity and quality of the generated molecules, when compared to uniform order dynamics.

Deep Learning · Generative Models and Autoencoders

Gaël Heck, Sylvie Le Hégarat-Mascle, Nicolas Lermé

Reassembling $N$ fragments in $n$-dimensional space is a shape reconstruction task that is invariant to global rigid motions. Training directly on $\mathcal{M}=\mathrm{SE}(n)^N$ can be ill-posed: standard losses penalize solutions that differ only by a global transform. Existing methods often address this with ad-hoc anchoring which breaks permutation invariance across fragments and can introduce biases that must be mitigated with extensive and costly data augmentation. We propose a geometric framework that enforces invariance by construction. First, a **Global Gauge Fixing** (GGF) strategy deterministically aligns configurations using an intrinsic generalized-inertia rule. Second, we introduce a **quotient-invariant Flow Matching objective** that operates via orthogonal projection onto the horizontal tangent bundle. This construction factors out global pose at each timestep, enabling the model to learn only shape-changing dynamics on the quotient space $\mathcal{M}/\mathrm{SE}(n)$. Our unified $\mathrm{SE}(n)$-invariant framework admits efficient closed-form 2D/3D instantiations and improves accuracy on polygonal jigsaw puzzles and 3D fracture reassembly benchmarks.

Deep Learning · Generative Models and Autoencoders

Denis Rakitin, Ivan Shchekotov, Viacheslav Meshchaninov, Dmitry Vetrov

Unpaired domain translation remains a challenging task due to the need of finding a balance between faithfulness and realism. In this paper, we propose a method called Regularized Distribution Matching Distillation (RDMD) that combines the best properties of Optimal Transport (OT) and diffusion-based domain translation methods. Instead of the conventional adversarial training, RDMD utilizes diffusion-based distribution matching, addressing the common shortcomings of OT methods and providing a strong initialization for the trained models. RDMD provides efficient one-step inference, explicitly controls the input-output alignment via regularization of the transport cost and maintains high faithfulness similar to the OT methods. We prove that in theory RDMD approximates the OT map and demonstrate its empirical performance on several tasks, including unpaired image-to-image translation in pixel and latent space and unpaired text detoxification. Empirical results show that RDMD achieves a comparable or better faithfulness-realism trade-off compared to the diffusion and OT baselines.

Deep Learning · Generative Models and Autoencoders

Bowen Xue, Zihan Min, Xingyang Li, Muyang Li, Yujun Lin, Zhekai Zhang, Haocheng Xi, Lvmin Zhang, Maneesh Agrawala, Jun-Yan Zhu 等

Diffusion models have become a dominant paradigm for high-quality generative modeling, while post-training is essential for adapting them to diverse downstream applications. However, post-training of large diffusion models is still challenging due to the prohibitive memory footprints and slow training speed, which existing parameter-efficient fine-tuning methods only partially address. To overcome these limitations, we propose FourTune, an efficient post-training framework for diffusion models based on an end-to-end W4A4G4 paradigm. FourTune introduces a triple-branch hybrid pipeline that augments the standard LoRA architecture with a frozen numerical stabilizer to isolate quantization-sensitive outliers, enabling stable training under native 4-bit computation. In addition, FourTune employs hardware-efficient block-wise quantization and customized fused kernels to support efficient quantized backpropagation and reduce memory bandwidth overhead. Across customization, reinforcement learning, and distillation tasks, FourTune matches the quality of full-precision fine-tuning. On FLUX.1-dev (12B), FourTune reduces memory overhead by $2.25\times$ and increases end-to-end training throughput by $2.27\times$ compared to BF16 LoRA.

Deep Learning · Generative Models and Autoencoders

Bumjun Kim, Dongjae Jeon, Moongyu Jeon, Albert No

Parallel decoding for diffusion LLMs (dLLMs) is difficult because each denoising step provides only token-wise marginal distributions, while unmasking multiple tokens simultaneously requires accounting for inter-token dependencies. We propose Dependency-Aware Parallel Decoding (DAPD), a simple, training-free decoding method that uses self-attention to induce a conditional dependency graph over masked tokens. At each iteration, edges in this graph capture strong token interactions, while non-edges indicate weak dependence. Parallel decoding is then reduced to selecting an independent set on the graph and unmasking the selected tokens in parallel. This avoids co-updating strongly coupled tokens without auxiliary models or retraining. Experiments on LLaDA and Dream show that DAPD improves the accuracy–steps trade-off over existing methods and enables more globally distributed parallel updates that better exploit the any-order generation capability of dLLMs.

Deep Learning · Generative Models and Autoencoders

Guanfang Dong, Luke Schultz, Negar Hassanpour, Chao Gao

Semantic-rich features from Vision Foundation Models (VFMs) have been leveraged to enhance Latent Diffusion Models (LDMs). However, raw VFM features are typically high-dimensional and redundant, increasing the difficulty of learning and reducing training efficiency for Diffusion Transformers (DiTs). In this paper, we propose Repack then Refine, a three-stage framework that brings the semantic-rich VFM features to DiT while further accelerating learning efficiency. Specifically, the RePack module projects the high-dimensional features onto a compact, low-dimensional manifold. This filters out the redundancy while preserving essential structural information. A standard DiT is then trained for generative modeling on this highly compressed latent space. Finally, to restore the high-frequency details lost due to the compression in RePack, we propose a Latent-Guided Refiner, which is trained lastly for enhancing the image details. On ImageNet-1K, RePack-DiT-XL/1 achieves an FID of 1.82 in only 64 training epochs. With the Refiner module, performance further improves to an FID of 1.65, significantly surpassing latest LDMs in terms of convergence efficiency. Our results demonstrate that packing VFM features, followed by targeted refinement, is a highly effective strategy for balancing generative fidelity with training efficiency.

General Machine Learning · Representation Learning

Xiaolei Liu, Binfeng Wang, Kaixin Gao, Shaoshuai Li

Cross-domain recommendation (CDR) serves as a pivotal solution to data sparsity and cold-start problems by transferring knowledge across distinct domains. However, existing approaches predominately rely on Euclidean embedding spaces, which suffer from a fundamental geometry-distribution mismatch: real-world user-item interactions typically exhibit power-law distributions and latent hierarchical structures that flat Euclidean spaces cannot accurately represent without significant distortion. This geometric limitation not only compromises representation quality but, more critically, hinders the effective disentanglement of domain-invariant user preferences from domain-specific interests, limiting transferability in low-overlap scenarios. To bridge this gap, we introduce the Mixed-Curvature Hyperbolic Variational Auto-Encoder (HVAE), a principled framework that unifies knowledge extraction and transfer within a hyperbolic manifold. By leveraging the exponential expansion capacity of hyperbolic geometry, HVAE naturally accommodates hierarchical data structures, enabling precise disentanglement of user intents without the need for strict domain overlap constraints. Furthermore, we propose a rigorous hyperbolic Wasserstein barycenter mechanism to align invariant distributions across heterogeneous domains. Extensive experiments on large-scale industrial and public datasets demonstrate that HVAE achieves superior performance, particularly in challenging scenarios with long-tail distributions and minimal domain overlap.

Deep Learning · Other Representation Learning

Manuel Hirth, Lukas Koberg, Nasser Jazdi, Enkelejda Kasneci

While diffusion models have advanced anomaly detection, their reliance on multi-step noise schedules introduces significant computational complexity. In this paper, we demonstrate that the generative capability of diffusion is not required for tabular one-class anomaly detection. We revisit core principles of denoising without targeting data generation and present a deep-learning approach that streamlines these objectives into a fixed-noise formulation. Unlike standard denoising autoencoders that rely on reconstruction error, our method utilizes a preconditioning with an explicit linear reference channel. We train a denoising predictor to recover clean samples from perturbed observations and derive anomaly scores from the expected deviation under repeated perturbations. We theoretically motivate this with a stability proxy by a first-order approximation rather than just distance to the manifold. On the well-established ADBench benchmark, our approach achieves improvements over existing methods of 1.22\% in AUCROC and 1.13\% in AUCPR, the most informative and threshold-independent metrics. Our approach emphasizes structural simplicity and efficiency, indicating that a single-step, stability-based objective outperforms complex generative schedules for tabular data.

Social Aspects · Accountability, Transparency, and Interpretability

Soyeon Kim, Seongwoo Lim, Kyowoon Lee, Jaesik Choi

Feature attribution is central to diagnosing and trusting deep neural networks, and Integrated Gradients (IG) is widely used due to its axiomatic properties. However, IG can yield unreliable explanations when the integration path between a baseline and the input passes through regions with noisy gradients. While Guided Integrated Gradients reduces this sensitivity by adaptively updating low-gradient-magnitude features, input-space guidance still produces intermediate inputs that deviate from the data manifold. To address this limitation, we propose **Manifold-Aligned Guided Integrated Gradients** (MA-GIG), which constructs attribution paths in the latent space of a pre-trained variational autoencoder. By ensuring that decoded intermediate images remain aligned with the data manifold, MA-GIG constrains gradient evaluation to statistically valid regions. Through qualitative and quantitative evaluations, we demonstrate that MA-GIG produces faithful explanations by aggregating gradients on path features proximal to the input. Consequently, our method suppresses off-manifold noise and outperforms prior path-based attribution methods across multiple datasets and classifiers.

Social Aspects · Accountability, Transparency, and Interpretability

Tang Li, Yanlin Chen, Mengmeng Ma, Xi Peng

Despite the high accuracy, Vision Transformer (ViT) predictions can be driven by spurious cues. Interpreting their inner workings is therefore essential for safe deployment. Sparse autoencoders (SAEs) shed light on decomposing language-model representations into concepts. However, adapting SAE-based interpretation to ViTs remains challenging due to limited control over concept coverage and subjective, non-scalable feature interpretation. To fill the gaps, motivated by stimulus-efficient probing in systems neuroscience, we propose ViSAE, a compact diagnostic toolbox for interpreting the internal mechanisms of ViTs. Specifically, we introduce a probing suite with 64K images and a 16K visually grounded concept vocabulary, improving concept coverage efficiency by 20× over ImageNet and interpretation accuracy by 28.7% over existing concept sets. We further develop an algorithm that automatically interprets SAE features at scale and causally traces cross-layer interactions to recover ViT inner workings using concept circuits. Our method supports auditing spurious correlations and failure modes, and boosts worst-group accuracy on WaterBirds by 48.2% through concept steering.

Deep Learning · Generative Models and Autoencoders

Haowen Zhong, Jiujun Cheng, Haowen Wang, Chao Wei, Lu Yang, Qichao Mao, Shangce Gao

Point cloud completion and generation are important across many 3D tasks, where both fidelity and sampling efficiency matter. Prevailing high-fidelity approaches rely on long sampling schedules, which incur substantial inference latency. Few-step alternatives typically use rectification or distillation, leading to multi-stage training pipelines and potential quality trade-offs. We present 3D MeanFlow (3DMF), a distillation-free model that performs one-step average-velocity transport for point cloud completion and generation. We optimize an instantaneous-average consistency objective and impose a shape-level constraint to stabilize training. Additionally, we introduce PointPlug, integrating completion into 3D object detectors and evaluating its impact. PointPlug uses adaptive selection that balances benefit and latency. Across standard benchmarks, 3DMF achieves one-step sampling with an order-of-magnitude speedup while maintaining competitive fidelity. On nuScenes and KITTI, inserting PointPlug improves all evaluated detectors under comparable settings.