论文检索

Deep Learning · Attention Mechanisms

Sumin Park, Seojin Kim, Noseong Park

Linear attention reformulates sequence modeling as recurrent state evolution, enabling efficient linear-time inference. Under the key–value associative paradigm, existing approaches restrict the role of the query to the readout operation, decoupling it from state evolution. We show that query-conditioned state readout induces a structured value prediction over accumulated memory that complements key-based retrieval. Based on this insight, we propose Q-Delta, a query-aware delta rule that integrates mixed key--query prediction errors into state evolution, enabling jointly corrective dynamics while preserving delta-rule efficiency. We establish stability guarantees for the resulting dynamics and derive a hardware-efficient chunkwise-parallel formulation with a custom Triton implementation. Empirical results demonstrate stable optimization, competitive throughput, and consistent improvements over strong baselines on language modeling and long-context retrieval tasks.

Deep Learning · Large Language Models

Sumin Park, Noseong Park

Mixture-of-Experts (MoE) scales model capacity efficiently by selectively routing inputs to a specialized subset of experts. However, input-expert specialization, the core motivation of MoE, critically depends on whether the router is actually aware of input structure. In practice, MoE routing is typically implemented as a shallow linear projection with limited awareness of input representation, which often leads to unstable and imbalanced routing. We propose STAR, a STructure-Aware Routing that rethinks MoE routing as a subspace learning problem by augmenting standard learnable routing with an evolving principal subspace that tracks dominant input structure via Generalized Hebbian Algorithm (GHA). By aligning routing decisions directly with input structure along with the task-supervision from learnable gate, STAR enables stable and balanced expert specialization without relying on auxiliary load-balancing losses. We evaluate STAR on controlled synthetic setup and large-scale language and vision tasks, where it consistently improves routing quality and downstream performance over strong MoE baselines. Moreover, optional test-time subspace updates further enhance routing robustness under distribution shifts.

Deep Learning · Foundation Models

Jianxiong Zhang, Bing Guo, Yuming Jiang, Haobo Wang, Bo An, Sean Du

Large reasoning models (LRMs) often generate long, seemingly coherent reasoning traces yet still produce incorrect answers, making hallucination detection challenging. Although trajectories contain useful signals, directly using trace text or vanilla hidden states for detection is brittle: traces vary in form and detectors can overfit to superficial patterns rather than answer validity. We introduce Answer-agreement Representation Shaping (ARS), which learns detection-friendly trace-conditioned representations by explicitly encoding answer stability. ARS generates counterfactual answers through small latent interventions, specifically, perturbing the trace-boundary embedding, and labels each perturbation by whether the resulting answer agrees with the original. It then learns representations that bring answer-agreeing states together and separate answer-disagreeing ones, exposing latent instability indicative of hallucination risk. The shaped embeddings are plug-and-play with existing embedding-based detectors and require no human annotations during training. Experiments demonstrate that ARS consistently improves detection and achieves substantial gains over strong baselines.

Theory · Everything Else

Naïl B. Khelifa, Richard E Turner, Ramji Venkataramanan

Machine learning models are increasingly trained or fine-tuned on synthetic data. Recursively training on such data has been observed to significantly degrade performance in a wide range of tasks, often characterized by a progressive drift away from the target distribution. In this work, we theoretically analyze this phenomenon in the setting of score-based diffusion models. For a realistic pipeline where each training round uses a combination of synthetic data and fresh samples from the target distribution, we obtain upper and lower bounds on the accumulated divergence between the generated and target distributions. This allows us to characterize different regimes of drift, depending on the score estimation error and the proportion of fresh data used in each generation. We also provide empirical results on synthetic data and images to illustrate the theory.

Applications · Computer Vision

Zhuang Xiong, Chen Zhang, Qingshan Xu, Wenbing Tao

Despite recent progress in calibration-free monocular SLAM via 3D vision foundation models, scale drift remains severe on long sequences. Motion-agnostic partitioning breaks contextual coherence and causes zero-motion drift, while conventional geometric alignment is computationally expensive. To address these issues, we propose VGGT-Motion, a calibration-free SLAM system for efficient and robust global consistency over kilometer-scale trajectories. Specifically, we first propose a motion-aware submap construction mechanism that uses optical flow to guide adaptive partitioning, prune static redundancy, and encapsulate turns for stable local geometry. We then design an anchor-driven direct Sim(3) registration strategy. By exploiting context-balanced anchors, it achieves search-free, pixel-wise dense alignment and efficient loop closure without costly feature matching. Finally, a lightweight submap-level pose graph optimization enforces global consistency with linear complexity, enabling scalable long-range operation. Experiments show that VGGT-Motion markedly improves trajectory accuracy and efficiency, achieving state-of-the-art performance in zero-shot, long-range calibration-free monocular SLAM.

Optimization · Large Scale, Parallel and Distributed

Insu Jang, Runyu Lu, Nikhil Bansal, Ang Chen, Mosharaf Chowdhury

Multimodal large language models (MLLMs) extend the capabilities of large language models (LLMs) by combining heterogeneous model architectures to handle diverse modalities like images and audio. However, this inherent heterogeneity in MLLM model structure and data types makes makeshift extensions to existing LLM training frameworks unsuitable for efficient MLLM training, especially in distributed training. In this paper, we present ModalGlue, an efficient distributed MLLM training framework that contemplates MLLM's unique characteristics in both model and data parallelization. ModalGlue introduces frozen-aware pipeline parallelism and workload-balanced context parallelism to improve MLLM training throughput. Our extensive evaluation shows that \name outperforms state-of-the-art solutions by $2.26\times$ on average in terms of MLLM training throughput.

Deep Learning · Foundation Models

Xinchen Zhang, Bowei Liu, Jiale Liu, Chufan Shi, Yizhen Zhang, Junhong Liu, Youliang Zhang, Zhiheng Li, Yujiu Yang, Ling Yang

Visual outcomes are increasingly central to multimodal large language models, making reliable and fine-grained verification essential for scaling generalist foundation models. In this work, we investigate ***multimodal meta-verification***, which leverages verifier-generated rationales rather than decision-only signals, and explore how to effectively incorporate meta-verification feedback into multimodal verifier training. We identify two key findings. First, symbolic verifier outputs (e.g., bounding boxes) outperform textual explanations as meta-verification rationales, enabling efficient rule-based reinforcement learning rewards while avoiding reliance on model-based rewards from auxiliary judge models. Second, decoupling reinforcement learning objectives for binary judgment and meta-verification substantially outperforms joint reward optimization, due to intrinsic differences in output structure and learning dynamics. Based on these insights, we train **OmniVerifier-M1**, a generalist visual verifier leveraging symbolic meta-verification and decoupled reinforcement learning. OmniVerifier-M1 provides robust verification and fine-grained error localization, and further enables **M1-TTS**, a verifier-driven agentic generation system achieving dynamic region-level self-correction. This approach paves the way for more reliable, interpretable, and fine-grained multimodal verification, supporting safer and more controllable foundation model deployment.

Deep Learning · Large Language Models

Xuan Wang, Liu Yilin, Fangxiang Feng, Caixia Yuan, Huixing Jiang, Xiaojie Wang

Vision-Language Models (VLMs) suffer from high inference latency due to long visual sequences. To enable efficient, on-demand utilization of visual information, we argue that visual necessity should be assessed by its semantic impact on the output distribution, rather than inferred from intermediate interaction signals such as attention weights. We propose a training-free framework based on token embedding subspace decomposition, which we term a prediction-conditioned Semantic Lens. Specifically, at fixed decoding intervals, we perform QR decomposition on the Top-K candidate token embeddings to construct an orthogonal semantic basis. We then introduce Semantic IImpact–Driven Visual Scheduling (SIVS), which measures how visual inputs impact model predictions by projecting visual-induced hidden-state variations onto this semantic lens. SIVS provides a geometrically grounded, impact-driven criterion for dynamic visual KV scheduling. Empirical results demonstrate that SIVS achieves ~87% visual KV compression while maintaining over 99% of model performance.

Xingyu Yu, Haoyu Wang, Haiyan Zhao, Fengxiang Wang, Xu Han

Quantization-aware training (QAT) is essential for extremely low-bit large language models (LLMs). Current QAT methods are mainly based on scalar quantization (SQ), which enables efficient optimization but suffers from severe performance degradation at 2-bit precision. On the other hand, vector quantization (VQ) provides substantially higher representational capacity, but its discrete codebook lookup prevents end-to-end training. We propose LC-QAT, a 2-bit weight-only VQ-QAT framework that represents quantized weights via a learned affine mapping over discrete vectors, which yields a high-quality PTQ initialization and enables fully differentiable end-to-end optimization without explicit codebook lookup in the training forward pass. This strong post-training initialization makes LC-QAT highly data-efficient. Experiments across diverse LLMs demonstrate that LC-QAT consistently outperforms state-of-the-art QAT methods while using only 0.1%–10% of the training data. Our results establish LC-QAT as a practical and scalable solution for extreme low-bit model deployment.

Deep Learning · Sequential Models, Time series

Michael Menezes, Anastasios Kyrillidis

While Mamba2's expanded state dimension enhances temporal modeling, it incurs substantial inference overhead that saturates bandwidth during autoregressive generation. Standard pruning methods fail to address this bottleneck: unstructured sparsity leaves activations dense, magnitude-based selection ignores runtime dynamics, and gradient-based methods impose prohibitive costs. We introduce GHOST (Grouped Hidden-state Output-aware Selection and Truncation), a structured pruning framework that approximates control-theoretic balanced truncation using only forward-pass statistics. By jointly measuring controllability and observability, GHOST rivals the fidelity of gradient-based methods without requiring backpropagation. As a highlight, on models ranging from 130M to 2.7B parameters, our approach achieves a 50% state-dimension reduction with approximately 1 perplexity point increase on WikiText-2. Code is available at https://anonymous.4open.science/r/mamba2_ghost-7BCB/.

General Machine Learning · Representation Learning

Boyang Li, Yulin Wu, Sizhe Xu, Nuoxian Huang, Zhonghang Yuan, Shangyi Guo, Shu Yang, Takahiro Yabe

Rotary Position Embedding (RoPE) is widely adopted in Transformer models, yet its extension to high-dimensional domains lacks a unified theoretical formulation. Most existing approaches either apply rotations independently along each axis or mix frequencies empirically, which limits cross-dimensional interactions and yields direction-dependent representations. To address these limitations, we propose *nD-RoPE*, a decomposition-free generalization of rotary embeddings to arbitrary dimensions. From a translation-invariant formulation in continuous Hilbert space, we derive a spectral condition for isotropy that requires treating positions and frequencies as coupled $n$-dimensional vectors. We instantiate this principle with a multi-scale regular simplex wave-vector design that provides uniform directional coverage with maximal symmetry. Experiments across images, videos, and point clouds demonstrate consistent performance gains and improved generalization in high-dimensional settings.

Deep Learning · Large Language Models

Qinghua Zhou, Ellina Aleshina, Andrey Lovyagin, Oleg Somov, Mikhail Seleznyov, Alexander Panchenko, Ivan Oseledets, Elena Tutubalina, Ivan Tyukin

The work presents an approach for addressing the challenge of robustness in Large Language Models (LLMs) to alterations and potential errors caused by semantically similar but textually different prompts. Recent works have shown that these kinds of prompt variations can significantly impact the performance of LLMs on tasks. The central question is: can LLMs' robustness to semantically-neutral prompt alterations be acquired without expensive retraining of the entire model? We address this question both theoretically and through experiments. Our theoretical analysis reveals a crucial factor impacting model robustness -- a systematic expected shift or perturbation-induced bias in neural network module outputs. Motivated by this analysis, we show that robustness can be achieved via a simple fine-tuning process: debiasing for robustness. We identify conditions when debiasing helps and when it does not, and demonstrate, through both theory and extensive experiments, that debiasing for robustness may indeed be a quick and efficient tool to enhance robustness and provide certification against random prompt perturbations.

Theory · Online Learning and Bandits

Mengtong Gao, Zhenhe Zhang, Jichen Li, Xia Xuanzhi, Wentao Zhou, Jing Chen

Two-sided matching markets are pervasive in numerous real-world applications, ranging from labor markets to online advertising. Recently, a rich line of research has studied the matching bandit problem, where participants learn their preferences through iterative interactions. However, existing works assume a static environment with fixed participants and require synchronized learning, in which all participants start simultaneously and have access to a global clock. In reality, matching markets are inherently dynamic: participants may enter and leave at arbitrary time steps without any global signal, leading to an uncoordinated scenario that render existing algorithms inapplicable. To address this challenge, we first investigate the one-sided learning setting under uncoordinated player arrivals, where only players need to learn their preferences. We propose the Way-SE algorithm, which achieves an $O(\frac{K^2 \log T}{\Delta_{\min}^2})$ regret through a distributed exploration mechanism that enables participants to implicitly coordinate exploration phases using only local clocks, without global synchronization. More importantly, we extend to the fully decentralized, dynamic two-sided learning setting where both sides need to learn their preferences and players may arrive or depart arbitrarily. We introduce Way-SE-2S, the first algorithm to achieve a sublinear regret $O(T^{\frac{K-1}{K}} \log T)$ in this challenging environment, without requiring global signals, restrictive preference structures, or observability of competing agents' outcomes. Our work provides the first theoretical guarantee for stable matching in fully decentralized and uncoordinated bandit markets.

Applications · Health / Medicine

Luke Nightingale, Joseph Tuersley, Scott Warchal, Andrea Cairoli, Jacob Howes, Cameron Shand, Andrew Powell, Darren Green, Amy Strange, Michael Howell

Phenotypic screening experiments produce many microscope images of cells under diverse perturbations, with biologically significant responses often subtle or difficult to identify visually. A central challenge is to extract image representations that distinguish activity from controls and group phenotypically similar perturbations. In this work we propose new adaptations of contrastive loss functions that incorporate experimental metadata as learned class vectors, and a geometrically inspired variant, called SPC, where class vectors are confined to the unit sphere and updated only by attractive terms (allowing more overlap of phenotypically similar classes). The approach is tested on two popular benchmarking datasets, BBBC021 and RxRx3-core; and we also evaluate performance on uncurated screens of HaCaT cells to gauge effectiveness in a realistic use-case scenario. We find we outperform prior methods across the three datasets and on a wide array of metrics measuring phenotype grouping, biological recall, drug-target interaction and mechanism-of-action inference. We also show we maintain this improved performance compared to models over 10x larger in parameter count, and that SPC can be used as an effective fine-tuning technique. The method is easy to implement and is well suited to settings with limited data or compute resources.

Deep Learning · Foundation Models

Youqi WU, Mohammad Jalali, Farzan Farnia

Multi-modal representation models such as CLIP, SigLIP, and their variants are widely used to represent data across multiple modalities in modern learning systems. While these models are commonly evaluated through downstream performance, the analysis of their structural differences in how multi-modal representations group data across modalities remains inadequate. In this work, we aim to identify modality pairs and sample subsets that induce divergent grouping behavior between two candidate embeddings. We propose \emph{Kernel Optimization for Discrepancy Analysis (KODA)}, a kernel method that constructs unified multi-modal kernels via modality-wise kernel multiplication and formulates discrepancy identification as an optimization problem that seeks components with high coherence under one embedding while constraining coherence under another. This formulation provides interpretable discrepancy structures associated with specific modality interactions. We establish finite-sample guarantees characterizing the effective reference sample size required for reliable analysis. To enable scalable computation in multi-modal settings, we develop a randomized low-dimensional approximation of joint kernels using random projections, including Random Fourier Features for shift-invariant kernels. Our empirical results indicate that KODA can identify consistent discrepancy structures across modalities.

Reinforcement Learning · Online

Hongze Tan, Zihan Wang, Jianfei Pan, Jinghao Lin, Hao Wang, Yifan Wu, Tao Chen, Zhihang Zheng, Tang, Haihua Yang

Reinforcement Learning (RL) is pivotal for enhancing Large Language Model (LLM) reasoning, yet mainstream algorithms such as GRPO and DAPO remain constrained by a coarse-grained credit assignment paradigm, where all tokens within the same response receive the identical reward. In this paper, we propose **Dynamic Entropy Weighting**, systematically define entropy-based weight ratios $\frac{H_{i,t}}{\sum_{k=1}^{n} H_{k,t}}$ and similar variants to redistribute rewards and get fine-grained rewards through two new algorithms: **Group Token Policy Optimization (GTPO)**, which assigns an entropy-weighted reward to each token and synthesizes token-specific advantage function to drive the model toward optimal path, and the analogous algorithm **Sequence-Level GRPO (GRPO-S)**, which admits a completely similar design at the sequence level. Unlike methods using entropy as mere regularization, GTPO and GRPO-S establish a new state-of-the-art on AIME and MATH 500, outperforming prior entropy-guided baselines and validating our weighting mechanism.

Deep Learning · Foundation Models

Yifan Liu, Zhiyuan Min, Zhenwei Wang, Junta Wu, Tengfei Wang, Yixuan Yuan, Yawei Luo, Chunchao Guo

We present WorldMirror, a unified feed-forward model for comprehensive 3D geometric prediction tasks. Unlike existing methods constrained to image-only inputs or customized for a specific task, our framework flexibly integrates diverse geometric priors, including camera poses, intrinsics, and depth maps, while simultaneously generating multiple 3D representations: dense point clouds, multi-view depth maps, camera parameters, surface normals, and 3D Gaussians. Remarkably, prior injection yields universal gains across all tasks, suggesting that input flexibility and multi-task prediction are mutually reinforcing. WorldMirror achieves state-of-the-art performance across diverse benchmarks from camera, point map, depth, and surface normal estimation to novel view synthesis, while maintaining the efficiency of feed-forward inference. Code and models will be publicly available.

Deep Learning · Graph Neural Networks

Sambaran Bandyopadhyay, Ananth Muppidi

Multi-hop Question Answering over Knowledge Graphs faces a critical challenge: traditional retrieve-then-read pipelines break differentiability, preventing the retriever from learning to bridge the semantic gap where intermediate nodes lack lexical overlap with the query. To address this, we propose RSF-GLLM, a framework decoupling differentiable graph reasoning from answer generation. Our Recurrent Soft-Flow (RSF) module employs a GRU-guided query updater to propagate continuous relevance scores, utilizing a dynamic gating mechanism to traverse semantically dissimilar bridge nodes via structural cues. We introduce flow sparsity regularization to theoretically guarantee convergence from soft probabilities to discrete reasoning paths. These paths are extracted and textualized to fine-tune a Large Language Model (LLM), ensuring generation is grounded in factual topology. Experiments on WebQSP and CWQ demonstrate that RSF-GLLM achieves competitive performance with superior inference efficiency compared to LLM based computationally expensive approaches.

General Machine Learning · Transfer, Multitask and Meta-learning

Wenju Sun, Qingyong Li, Tiancheng Li, Yangliao Geng, Albert Boyang Li

Model merging offers an efficient solution for integrating task-specific knowledge from multiple fine-tuned models. Most existing approaches focus on manipulating the difference vectors between fine-tuned and pre-trained weights, often overlooking the generalization capabilities inherent in the pretrained parameters themselves. In this work, we revisit the role of pretrained weights in model merging and investigate their efficacy from a subspace perspective. We find that the components of pretrained weights residing in the core subspace—defined by the dominant singular vectors—are essential for maintaining generalization across diverse tasks. Specifically, we present empirical evidence that pretrained weights are nearly first-order stationary and exhibit predominantly non-negative curvature within this core subspace with respect to multi-task loss landscapes, indicating near-optimality. These findings suggest that task-specific adaptations should be injected primarily into the orthogonal complement of the core subspace, thereby preserving the generalization properties of the pretrained model. Extensive experiments on vision and vision-language tasks show that this subspace-aware strategy consistently yields improvements over state-of-the-art training-free merging methods, including Task Arithmetic, LOT Merging, ISO, and TSV.

Applications · Health / Medicine

Haonan Zhang, Qing Wu, Xuanyu Tian, Bowen Li, Yuyao Zhang, Hongjiang Wei

Implicit Neural Representation (INR) has emerged as a powerful paradigm for continuous MRI reconstruction. However, standard unsupervised INR requires time-consuming optimization from scratch for each scan, hindering clinical deployment. This work presents IPOD, a Reference-Free Meta-Learning framework designed to learn generalized parameter initializations for INR directly from undersampled data. Distinct from conventional meta-learning that relies on fully-sampled ground truth, IPOD operates in an inverse-problem-driven manner, leveraging diverse reconstruction tasks with varying sampling patterns to capture a robust prior. Furthermore, we introduce an adaptive meta-update strategy modulated by task-specific performance to ensure optimal parameter distribution for diverse anatomical structures. Extensive experiments demonstrate that IPOD provides a superior initialization that enables rapid adaptation and achieves high-fidelity reconstruction across various imaging protocols, significantly outperforming existing INR baselines. By eliminating the dependence on reference images, IPOD offers a scalable and efficient solution for a wide range of imaging inverse problems. Code and data available at: https://anonymous.4open.science/r/iPod-2C60