论文检索

Applications · Chemistry, Physics, and Earth Sciences

Muhammad Umer Sheikh, Hassan Abid, Khawar shehzad, Ufaq Khan, Muhammad Haris Khan

Climate change research increasingly requires AI systems that reason across text, dynamic visual content, and scientific figures, yet existing climate QA benchmarks are small, mostly textual, and cover a narrow range of models. We introduce MMClima, a large-scale multimodal climate question answering framework with over 104k expert-validated question–answer pairs spanning articles, video transcriptions, and figures across five core climate science domains. MMClima is constructed via automated claim extraction and QA synthesis with human-in-the-loop validation to ensure both scale and reliability. Using MMClima, we benchmark state-of-the-art multimodal language models on tasks requiring factual recall, visual interpretation, and cross-modal synthesis. We additionally fine-tune on the textual split to produce mmclima-70b-txt, a domain-adapted baseline that outperforms strong open- and closed-source models on textual QA. We release the dataset, evaluation pipeline, fine-tuned model weights, and data creation framework to support standardized multimodal evaluation for climate science.

Deep Learning · Large Language Models

Paul Jünger, Justin Lovelace, Linxi Zhao, Dongyoung Go, Kilian Weinberger

Diffusion language models offer fast, parallel decoding via non-autoregressive generation and uncertainty-aware denoising, yet these properties remain underexplored for retrieval. We propose *Self-Augmenting Retrieval for Diffusion Language Models*, a dynamic framework that uses intermediate diffusion states to refine retrieval throughout the denoising trajectory. At each iteration, we query an external corpus with the partially denoised text, retrieve additional evidence, and condition subsequent denoising steps on the updated context. This tightly couples retrieval to the diffusion process: high-confidence tokens guide retrieval early, while uncertain spans are completed after new evidence is incorporated. Experiments with DREAM-7B, a discrete diffusion language model, on open-domain question answering benchmarks show significant improvements in answer accuracy over static question-only retrieval, while achieving 2--6$\times$ higher throughput than autoregressive baselines, demonstrating that diffusion decoding offers a compelling paradigm for efficient, high-quality retrieval-augmented generation.

Deep Learning · Large Language Models

Hong-Yu Chen, Venkat Ganti, Jerry Yao-Chieh Hu, Hude Liu, Han Liu

We show that Chain-of-Thought (CoT) enables a fixed single-layer transformer to efficiently approximate the training process of an $N$-layer feed-forward network in-context. Since FFNs are universal approximators, this result provides strong theoretical evidence for the expressive power of CoT. Specifically, we improve the computational cost of the prior best in-context result [Wu et al., ICML 2025] by $O(N)$. Building on the insight of the recursive nature of CoT, we reuse the single-layer transformer autoregressively instead of stacking the same transformer blocks to perform multiple In-Context Gradient Descent (ICGD) updates. The key novelty is a dynamic-masking scheme: at each CoT step, the attention heads are forced to see only the tokens needed to compute the result of the current forward or backpropagation update. This selective reuse contrasts with earlier ICGD proofs for neural network optimization, which must carry all information through every layer simply because a later gradient update might need it. Our numerical validations backup our theory.

Deep Learning · Large Language Models

Zhendong Huang, Hengjie Cao, Fang DONG(董方), Ruijun Huang, Mengyi Chen, Yifeng Yang, Xin Zhang, Anrui Chen, Mingzhi Dong, Yujiang Wang 等

Gradient signals in LLM training are highly anisotropic: recurrent linguistic structure concentrates energy into a small set of dominant spectral directions, while context-specific information resides in a long tail. We show that this spike–tail separation persists throughout training, with the spike occupying only about 1.5% of directions yet dominating optimizer statistics. This dominance suppresses tail learning by contracting tail updates through second-moment normalization and tightening the globally stable learning-rate bound. Motivated by this analysis, we propose \textit{Spectra}, a spike-aware optimizer that suppresses the dominant low-rank spike subspace without amplifying the noise-sensitive spectral tail. Spectra tracks the spike subspace via cached, warm-started power iteration and applies low-rank spectral shaping with negligible overhead and substantially reduced optimizer-state memory. On LLaMA3-8B trained on 50B tokens, Spectra reaches the same target loss 30% faster than AdamW, reduces per-step end-to-end overhead by 0.7%, cutting optimizer-state memory by 49.25%, and improves average downstream accuracy by 1.62%. Compared to Muon, Spectra is $5.1\times$ faster in optimizer processing time, achieves a lower final loss, and improves average accuracy by 0.66%. Spectra's Megatron integration is released publicly (https://tinyurl.com/29n4vv5f).

Wei Liu, Hongkai Liu, Zhiying Deng, Yee-Whye Teh, Wee Sun Lee

LLM parameter editing methods commonly rely on computing an ideal target hidden-state at a target layer (referred as anchor point) and distributing the target vector to multiple preceding layers (commonly known as backward spreading) for cooperative editing. Although widely used for a long time, its underlying basis have not been systematically investigated. In this paper, we first conduct a systematic study of its foundations, which helps clarify its capability boundaries, practical considerations, and potential failure modes. Then, we propose a simple and elegant alternative that replaces backward spreading with forward-propagation. Instead of optimizing the target at the last editing layer, we optimize the anchor point at the first editing layer, and then propagate it forward to obtain accurate and mutually compatible target hidden-states for all subsequent editing layers. This approach achieves the same computational complexity as existing methods while producing more accurate layer-wise targets. Our method is simple, without interfering with either the computation of the initial target hidden state or any other components of the subsequent editing pipeline, and thus constituting a benefit for a wide range of LLM parameter editing methods.

Probabilistic Methods · Monte Carlo and Sampling Methods

Zonghao Chen, Heishiro Kanagawa, Francois-Xavier Briol, Chris J Oates, Lester Mackey

Several important learning tasks can be formulated as minimizing an entropy-regularized objective over an appropriate space of probability distributions. Mean-field Langevin dynamics (MFLD) facilitate computation in this general context, casting the minimizer as the invariant distribution of a McKean--Vlasov process, which can be numerically discretized using $N$ particles and thus simulated. However, simulating this interacting particle system has computational complexity $\mathcal{O}(N^2)$. Motivated by recent research into \emph{kernel thinning}, we propose \texttt{KT-MFLD}, in which each particle interacts only with a coreset of size $\mathcal{O}(N^{\frac{1}{2}})$. \texttt{KT-MFLD} thus reduces the computational complexity to $\mathcal{O}(N^{\frac{3}{2}})$ while, under mild regularity conditions, achieving the same convergence guarantees (up to logarithmic factors) as MFLD. Our theoretical analysis is empirically confirmed on tasks including the training of student-teacher neural networks, quantization with maximum mean discrepancy, and computation of predictively-oriented posteriors in a post-Bayesian framework.

Applications · Computer Vision

Haoyu Xiong, Chengchao Wang, ZhongQiang Wang, Huang He, Qiuxia Yang, Zhengpeng Zhao, Yuanyuan Pu

Test-Time Adaptation (TTA) empowers pre-trained models to adapt online to distribution shifts during inference, but such online updates often become unstable in long-horizon deployments. Prevailing approaches attribute this failure to error accumulation from noisy pseudo-labels, relying on heuristics to gate which samples are used for updates. We argue that this statistical view is insufficient: the problem lies not only in the quality of samples but also in the directionality of their gradients. In this work, we identify a geometric failure mode termed manifold erosion. Through spectral analysis, we find that reliable gradients concentrate in a stable low-rank subspace, while gradients from confident mispredictions are high-rank yet exhibit a persistent directional leakage into this protected subspace. This leakage can accumulate coherently and gradually erode core representations, eventually leading to collapse. To address this, we propose Manifold-Aware Gradient Projection (MGP), a geometric intervention that tracks the dominant subspace online and projects gradients onto its orthogonal complement. By blocking the leakage path, MGP decouples stability from plasticity. Extensive experiments on diverse TTA benchmarks demonstrate the long-horizon stability of our method, whereas prior methods often fail.

General Machine Learning · Evaluation

Yiqun Chen, Sizhu Lu, Sijia Li, Moran Guo, Shengyi Li

Large language models (LLMs) are increasingly used as automatic evaluators of generative AI outputs, a paradigm often referred to as "LLM-as-a-judge." In practice, LLM judges are imperfect predictions for the underlying truth and can exhibit systematic, non-random errors. Two main approaches have recently been proposed to address this issue: (i) *direct measurement-error correction* based on misclassification models such as Rogan--Gladen-style estimators, and (ii) *surrogate-outcome approaches* such as prediction-powered inference (PPI), which correct bias by calibrating prediction residuals on a small set of gold-standard human labels. In this paper, we systematically study the performance of these two approaches for estimating mean parameters (e.g., average benchmark scores or pairwise win rates). Leveraging tools from semiparametric efficiency theory, we unify the two classes of estimators by deriving explicit forms of *efficient influence function*-based efficient estimators and characterize conditions under which PPI-style estimators attain strictly smaller asymptotic variance than measurement-error corrections. We verify our theoretical results through simulations and demonstrate the methods on a real-data example using our open-source software package for performing the calibration.

Applications · Computer Vision

Jun Lin, Jiayu Ding, Xiangtian Si, Xitong Cao, Lixin Hong, Zhang Chen, Chenxi Lv, Wenqian Wang

Existing 3D Visual Question Answering (3D-VQA) methods rely on generative paradigms, producing ambiguous descriptions that hinder deterministic decision-making. We introduce 3D Scene Assertion Verification, a task requiring models to verify natural language assertions in 3D scenes with strict binary judgments. To enable rigorous evaluation, we present 3DSAV, the first large-scale diagnostic benchmark comprising 22.5k samples tailored for this objective. We observe that current 3D-VQA models struggle in this setting due to a lack of specialized reasoning mechanisms. To address this, we propose DualLPSS. This framework uses a dual-stage routing mechanism to enable type-aware cross-modal fusion and scene-guided assertion focusing. Extensive experiments show that DualLPSS achieves state-of-the-art performance on 3DSAV, distinguishing itself by correctly handling complex logical assertions where baselines fail. The code and dataset will be made publicly available.

General Machine Learning · Unsupervised and Semi-supervised Learning

Erell Gachon, Jérémie Bigot, Elsa Cazelles

A common approach to perform PCA on probability measures is to embed them into a Hilbert space where standard functional PCA techniques apply. While convergence rates for estimating the embedding of a single measure from $m$ samples are well understood, the literature has not addressed the setting involving multiple measures. In this paper, we study PCA in a double asymptotic regime where $n$ probability measures are observed, each through $m$ samples. We derive convergence rates of the form $n^{-1/2} + m^{-\alpha}$ for the empirical covariance operator and the PCA excess risk, where $\alpha>0$ depends on the chosen embedding. This characterizes the relationship between the number $n$ of measures and the number $m$ of samples per measure, revealing a sparse (small $m$) to dense (large $m$) transition in the convergence behavior. Moreover, we prove that the dense-regime rate is minimax optimal for the empirical covariance error. Our numerical experiments validate these theoretical rates and demonstrate that appropriate subsampling preserves PCA accuracy while reducing computational cost.

Deep Learning · Sequential Models, Time series

Julien Brandoit, Arthur Fyon, Damien Ernst, Guillaume Drion

Sequence learning is dominated by Transformers and parallelizable recurrent neural networks such as state-space models, yet learning long-term dependencies remains challenging, and state-of-the-art designs trade power consumption for performance. The Bistable Memory Recurrent Unit (BMRU) was introduced to enable hardware–software co-design of ultra-low power RNNs: quantized states with hysteresis provide persistent memory while mapping directly to analog primitives. However, BMRU performance lags behind parallelizable RNNs on complex sequential tasks. In this paper, we identify gradient blocking during state updates as a key limitation and propose a cumulative update formulation that restores gradient flow while preserving persistent memory, creating skip-connections through time. This leads to the Cumulative Memory Recurrent Unit (CMRU) and its relaxed variant, the $\alpha$CMRU. Experiments show that the cumulative formulation dramatically improves convergence stability and reduces initialization sensitivity. The CMRU and $\alpha$CMRU match the performance of Linear Recurrent Units (LRUs) and minimal Gated Recurrent Units (minGRUs) on standard benchmarks at small model sizes, while the CMRU retains quantized states, persistent memory, and noise-resilient dynamics essential for analog implementation.

Deep Learning · Large Language Models

Jiaru Zou, Xiyuan Yang, Ruizhong Qiu, Gaotang Li, Katherine Tieu, Pan Lu, Ke Shen, Hanghang Tong, Yejin Choi, Jingrui He 等

Multi-agent systems (MAS) extend large language models (LLMs) from independent single-model reasoning to coordinative system-level intelligence. While existing LLM agents depend on text-based mediation for reasoning and communication, we take a step forward by enabling models to collaborate directly within the continuous latent space. We introduce LatentMAS, an end-to-end training-free framework that enables pure latent collaboration among LLM agents. In LatentMAS, each agent first performs auto-regressive latent thoughts generation through last-layer hidden embeddings instead of text. Then, a shared latent working memory preserves and transfers each agent's internal representations and latent thoughts, ensuring lossless information exchange without re-encoding. We provide detailed theoretical analyses showing that LatentMAS achieves higher expressiveness and lossless information preservation with lower overall complexity than standard text-based MAS. In addition, empirical evaluations across 9 comprehensive benchmarks spanning math and science reasoning, commonsense understanding, and code generation show that LatentMAS outperforms advanced single agents and text-based MAS baselines, achieving up to 14.6\% higher accuracy, reducing output token usage by 70.8\%-83.7\%, and providing 4$\times$-4.3$\times$ faster end-to-end inference. These results demonstrate that our new latent collaboration framework enhances system-level reasoning quality while providing consistent efficiency gains.

Deep Learning · Generative Models and Autoencoders

Rajalaxmi Rajagopalan, Debottam Dutta, Yu-Lin Wei, Romit Roy Choudhury

Imagine Alice has a specific image $x^\ast$ in her mind, say, the view of the street in which she grew up during her childhood. To generate that exact image, she guides a generative model with multiple rounds of prompting and arrives at an image $x^{p*}$. Although $x^{p*}$ is reasonably close to $x^\ast$, Alice finds it difficult to close that gap using language prompts. This paper aims to narrow this gap by observing that even after language has reached its limits, humans can still tell when a new image $x^+$ is closer to $x^\ast$ than $x^{p*}$. Leveraging this observation, we develop **MultiBO** (Multi-Choice Preferential Bayesian Optimization) that carefully generates $K$ new images as a function of $x^{p*}$, gets preferential feedback from the user, uses the feedback to guide the diffusion model, and ultimately generates a new set of $K$ images. We show that within $B$ rounds of user feedback, it is possible to arrive much closer to $x^\ast$, even though the generative model has no information about $x^\ast$. Qualitative scores from $30$ users, combined with quantitative metrics compared across $5$ baselines, show promising results, suggesting that multi-choice feedback from humans can be effectively harnessed for personalized image generation.

Deep Learning · Large Language Models

Liyuan Mao, Le Yu, Jing Zhou, Chujie Zheng, Bowen Yu, Chang Gao, Shixuan Liu, An Yang, Weinan Zhang, Junyang Lin

In this work, we reveal that Large Language Models (LLMs) possess intrinsic behavioral plasticity—akin to chameleons adapting their coloration to environmental cues—that can be *exposed* through token-conditional generation and *stabilized* via reinforcement learning. Specifically, by conditioning generation on carefully selected token prefixes sampled from responses exhibiting desired behaviors, LLMs seamlessly adapt their behavioral modes at inference time (e.g., switching from step-by-step reasoning to direct answering) without retraining. Based on this insight, we propose **To**ken-**Co**nditioned **R**einforcement **L**earning (**ToCoRL**), a principled framework that leverages RL to internalize this chameleon-like plasticity, transforming transient inference-time adaptations into stable and learnable behavioral patterns. ToCoRL guides exploration with token-conditional generation and keep enhancing exploitation, enabling emergence of appropriate behaviors. Extensive experiments show that ToCoRL enables precise behavioral control without capability degradation. Notably, we show that large reasoning models, while performing strongly on complex mathematics, can be effectively adapted to excel at factual question answering, which was a capability previously hindered by their step-by-step reasoning patterns.

Social Aspects · Security

Wenqi Chen, Ziyan Zhang, Bin Wang, Lin Liu, Hengheng Zhang, Zhengsu Chen

While Large Language Models (LLMs) excel in code generation, they remain prone to replicating subtle yet critical vulnerabilities endemic to their training data. Current alignment techniques, such as Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), typically apply coarse-grained optimization at the sequence level. This approach often fails to address the localized nature of security flaws, where a single incorrect token choice can compromise an entire program. To bridge this gap, we introduce Tree-like Self-Play (TSP), a framework that reframes secure code generation as a fine-grained sequential decision process. Unlike standard methods that blindly maximize likelihood, TSP constructs a decision tree where the model explores branching trajectories—generating both secure "golden paths" and vulnerable variants. By treating code generation as a self-play game, the model learns to strictly discriminate against its own localized errors. This provides a dense, on-policy learning signal that forces self-correction precisely at the critical decision nodes where vulnerabilities typically emerge. Our experiments demonstrate that TSP fundamentally enhances model reliability. In Python security benchmarks, TSP boosts CodeLlama-7B’s pass rate (SPR@1) to 75.8\%, significantly outperforming SFT (57.0\%) and unstructured self-play baselines. Crucially, TSP induces robust out-of-distribution generalization: the model not only reduces vulnerabilities in unseen categories (CWEs) by 24.5\% but also successfully transfers security principles learned from C/C++ to diverse languages, including Python, Go, and JavaScript. This suggests that TSP does not merely memorize patches, but internalizes abstract, language-agnostic security logic.

Applications · Computer Vision

Junyi Hu, Tian Bai, Fengyi Wu, Wenyan Li, Zhenming Peng, Yi ZHANG

Open-vocabulary grounding requires accurate vision-language alignment under weak supervision, yet existing methods either rely on global sentence embeddings that lack fine-grained expressiveness or introduce token-level alignment with explicit supervision or heavy cross-attention designs. We propose \textbf{ExpAlign}, a theoretically grounded vision-language alignment framework built on a principled multiple instance learning formulation. ExpAlign introduces an Expectation Alignment Head that performs attention-based soft MIL pooling over token-region similarities, enabling implicit token and instance selection without additional annotations. To further stabilize alignment learning, we develop an energy-based multi-scale consistency regularization scheme, including a Top-K multi-positive contrastive objective and a Geometry-Aware Consistency Objective derived from a Lagrangian-constrained free-energy minimization. Extensive experiments show that ExpAlign consistently improves open-vocabulary detection and zero-shot instance segmentation, particularly on long-tail categories. Most notably, it achieves 36.2 AP$_r$ on the LVIS minival split, outperforming other state-of-the-art methods at comparable model scale, while remaining lightweight and inference-efficient.

Deep Learning · Large Language Models

Yuchen Xian, Yang He, Yunqiu Xu, Yi Yang

Speculative decoding (SD) addresses the high inference costs of large language models (LLMs) by having lightweight drafters generate candidates for large verifiers to validate in parallel. Existing draft-verify methods use binary decisions: accept or fully recompute. Yet we find that many rejected tokens can be verified correctly by a slim submodel derived from the full verifier via intra-model routing, instead of the full verifier. This motivates our slim-verifier to handle tokens requiring moderate verification resources, reducing expensive large-model calls. We propose **V**erification via **I**ntr**a**-Model Routing for **S**peculative **D**ecoding (VIA-SD), a multi-tier framework using a routed slim-verifier. Draft tokens are processed hierarchically: direct acceptance for high-confidence cases, slim-verifier regeneration for medium-confidence cases, and full-model verification for uncertain cases. Across summarization, translation, reasoning, QA, and coding tasks on encoder-decoder and decoder-only model families, VIA-SD consistently lowers rejection rates (0.1–0.22) and achieves 10–20\% speedup over state-of-the-art SDs. Compared to decoding without drafting, VIA-SD provides 2.5-3× acceleration while improving output quality. Moreover, VIA-SD is compatible with existing SD frameworks without modifying their training procedures. Our results establish multi-tier SD as a general paradigm for scalable and efficient LLM inference. Our code will be publicly available.

Optimization · Discrete and Combinatorial Optimization

Sebastian Lüderssen, Ioana-Oriana Bercea, Stefan Neumann

For many NP-hard optimization problems, strong theoretical inapproximability results exist. However, in practice, heuristics regularly outperform these pessimistic worst-case results on real-world datasets. Assessing the quality of these algorithms' outputs is often difficult since we lack good lower bounds on the optimal solution. In this paper, we present efficient algorithms for computing lower bounds on the optimal solutions for correlation clustering, which is a popular problem in social-network analysis. Our lower bounds allow us to provide empirical certificates that bound the solution quality of practical algorithms by obtaining instance-specific approximation ratios. Our main technical contribution is an algorithm that approximates an LP relaxation of a related triangle covering problem in near-linear time on sparse graphs; the algorithm is based on the multiplicative weights update framework and runs on graphs with millions of edges in a few minutes. For the concrete problem of correlation clustering, our lower bounds certify that state-of-the-art heuristics achieve almost optimal approximation ratios of 0.94 for the agreement version and 1.97 for the disagreement version (averaged over 7 real-world datasets). We also show similar results for the fundamental max-cut problem.

Reinforcement Learning · Everything Else

Heman Shakeri

Controlling spreading processes on networks such as epidemics, information cascades, product adoption, requires policies that perform on realistic stochastic dynamics, not just tractable approximations. Yet policies trained on standard simplifications (mean-field ODEs, Markovian dynamics) suffer severe performance degradation at deployment. We trace this sim-to-real gap to three theoretical pathologies: Optimism Bias, where deterministic approximations systematically underestimate variance via Jensen's inequality; Hub Blindness, where global state aggregation obscures the super-spreaders driving scale-free networks; and the Valley of Death, where mean-value critics fail to navigate the bimodal nature (extinction vs. viral) of cascade outcomes. We resolve these challenges through two synergistic contributions. First, the Stratified Mean-Field Observer partitions nodes by influence tier, preserving hub dynamics at $O(N)$ cost while producing fixed-dimensional observations that enable zero-shot transfer across network scales and topologies. Second, we demonstrate that Distributional RL via Truncated Quantile Critics is essential for risk-aware control of bimodal cascades. Trained on a GPU-accelerated simulator supporting non-Markovian renewal dynamics, our approach achieves $59\times$ improvement over Markovian baselines and robust zero-shot transfer to real-world social networks (Facebook, Twitter, YouTube), effectively closing the simulation-to-reality gap.

General Machine Learning · Representation Learning

Huaihai Lyu, Chaofan Chen, Mingyu Cao, Yuheng Ji, Changsheng Xu

Achieving robust generalization from limited data is a central challenge in embodied intelligence. Prevailing methods fail by regressing absolute coordinates, which violates the principle of general covariance. Theoretically, this conflates the intrinsic task geometry with rigid execution patterns, binding policies to specific motion styles and fixed speeds. To resolve this, we propose the Generalized Action Manifold (GAM) framework that enforces general covariance through structural disentanglement. Specifically, GAM constructs the manifold by enforcing invariance across two orthogonal dimensions: (1) Temporal Invariance, utilizing an Arc-Length Parameterizer to orthogonalize the spatial path geometry from temporal dynamics, ensuring robustness to velocity variations; (2) Geometric Invariance, where a Schema-Affine-Factorization mechanism maps trajectories to canonical “world lines” in the Lie-algebraic tangent space. This distinguishes invariant topological schemas from affine modulations, ensuring spatial generalizability. By integrating GAM within a structured Vision-Language-Action (VLA) architecture, we expand sparse training data into a continuous, valid action manifold. Empirical results demonstrate that GAM enables superior transfer and robustness capabilities, significantly outperforming geometry-agnostic baselines.