论文检索

Applications · Computer Vision

Seojeong Park, Jiho Choi, Junyong Kang, Seonho Lee, Jaeyo Shin, Hyunjung Shim

Recent multimodal large language models have demonstrated strong reasoning ability, yet their reliability as automated evaluators remains limited by a critical weakness: when visual evidence conflicts with textual cues, MLLM judges tend to reward plausible narratives over perceptually correct answers. We identify and systematically analyze this phenomenon, which we term \textit{Perceptual Judgment Bias}. Through controlled visual perturbations, existing multimodal judges frequently anchor on the response text instead of their own visual perception, leading to inconsistent and non-verifiable evaluations. To address this issue, we introduce the \textit{Perceptually Perturbed Judgment Dataset}, which constructs minimally edited counterfactual responses that isolate perceptual errors and enable verifiable supervision. Building on this dataset, we develop a unified training framework that combines a structured GRPO-based reward with a batch-ranking objective, achieving coherent global ordering without explicit pairwise labels. Experiments across diverse MLLM-as-a-Judge benchmarks show that our approach substantially improves perceptual fidelity, ranking coherence, and alignment with human evaluation. Our results establish a scalable and generalizable pathway for training multimodal judges that are perceptually grounded, interpretable, and robust to visual–reasoning conflicts.

General Machine Learning · Evaluation

Wonjoong Kim, Sangwu Park, Yeonjun In, Sein Kim, Dongha Lee, Chanyoung Park

Driven by recent advancements in tool-augmented Large Language Model (LLM) agents, comprehensive benchmark datasets for evaluating these tool-augmented agents are being actively developed. Although these benchmarks incorporate increasingly complex user requests and a diverse array of tools, the evaluation methods for most of them remain limited to answer matching. However, as the number of steps required to resolve a user request increases, a proper evaluation of an agent's performance must go beyond the final answer to also assess the problem-solving trajectory, including previously ignored aspects such as efficiency, hallucinations, and adaptivity. The most straightforward method for evaluating these aspects is to compare the trajectory of the agent with a ground-truth trajectory, but this approach is fundamentally limited since annotating all possible ground-truth trajectories is prohibitively expensive. To address these significant gaps, we introduce TRACE, a framework for the multi-dimensional evaluation of tool-augmented LLM agent performance. By incorporating evidence store, TRACE enables a multi-faceted analysis and evaluation of an agent's reasoning trajectory, eliminating the need for a predefined ground-truth trajectory. To validate our framework, we develop a new meta-evaluation dataset by augmenting existing benchmarks with diverse and flawed trajectories, each labeled with multi-faceted performance scores. Our results confirm that TRACE accurately evaluates these complex behaviors in a scalable and cost-effective manner, even with small open-source LLMs.

Deep Learning · Large Language Models

Haocheng Xi, Harman Singh, Yuezhou Hu, Coleman Hooper, Rishabh Tiwari, Aditya Tomar, Wonjun Kang, Minjae Lee, Michael Mahoney, Chenfeng Xu 等

Block-wise diffusion language models (DLMs) generate multiple tokens in parallel, offering a promising alternative to autoregressive decoding. However, their inference efficiency remains bottlenecked by memory-bound attention in long-context scenarios. Naïve sparse attention is ineffective for DLMs due to the KV inflation problem: different queries select different prefix positions, causing the union of accessed KV pages to remain large. To address this challenge, we observe that block-wise diffusion exhibits locality of representation changes across denoising steps: only a small fraction of tokens (active tokens) undergo significant hidden-state updates, while most tokens (stable tokens) remain nearly unchanged. Based on this insight, we propose LoSA (Locality-aware Sparse Attention), which reuses cached prefix-attention results for stable tokens and applies sparse attention only to active tokens with large representation changes. This design reduces the number of queries contributing to the union of KV indices, substantially shrinking the KV pages that must be loaded. Across multiple block-wise DLMs and reasoning benchmarks, LoSA preserves near-dense accuracy while significantly improving efficiency, achieving up to 4.14× speedup over dense attention on RTX A6000 GPUs. LoSA also delivers up to 5% average improvement over baselines across all datasets and configurations, demonstrating the effectiveness of the proposed method.

Applications · Time Series

Jihye Na, Patara Trirat, Chanyoung Park, Jae-Gil Lee

Recent studies have attempted to fine-tune time-series foundation models to enhance a target dataset's forecasting performance. However, these approaches proceed without a clear criterion for identifying complex datasets that require fine-tuning due to performance degradation in zero-shot forecasting. To distinguish datasets that are more challenging from standard benchmarks, we introduce data-driven temporal complexity and multichannel complexity. *Temporal complexity* captures the difficulty of identifying distinct patterns by quantifying spectral entropy in the frequency domain, while *multichannel complexity* captures inter-channel dependencies by measuring the channel information flow impacting predictive uncertainty. These metrics serve as *effective proxies for performance gains* achievable through fine-tuning. Based on the two metrics, we develop *Time-PEFT*, a parameter-efficient fine-tuning framework that incorporates a frequency adapter for top-$k$ filtering and a channel adapter for multichannel modeling. *Time-PEFT* is shown to significantly improve forecasting performance by up to 2.51 times compared with existing fine-tuning techniques on complex datasets.

Reinforcement Learning · Batch/Offline

Nazim Bendib, Nicolas Perrin-Gilbert, Olivier Sigaud

Offline zero-shot reinforcement learning (RL) aims to learn agents that optimize unseen reward functions without additional environment interaction. The standard approach to this problem trains task-conditioned policies by sampling task vectors that define linear reward functions over learned state representations. In most existing algorithms, these task vectors are randomly sampled, implicitly assuming this adequately captures the structure of the task space. We argue that doing so leads to suboptimal zero-shot generalization. To address this limitation, we propose extracting task vectors directly from the offline dataset and using them to define the task distribution used for policy training. We introduce a simple and general reward function extraction procedure that integrates into existing offline zero-shot RL algorithms. Across multiple benchmark environments and baselines, our approach improves zero-shot performance by an average of 20%, highlighting the importance of principled task sampling in offline zero-shot RL.

Deep Learning · Foundation Models

Jingru Fei, Kun Yi, Alex Wang, Qingsong Wen, Xiangxiang Zhu, Wei Fan

Time series foundation models rely on large-scale pretraining over diverse datasets across domains, yet their heterogeneity in temporal patterns could hinder the effectiveness of training and learning transferable time series representations. Inspired a fundamental concept, normalized power spectral density (PSD) in signal processing, we assume harmonizing datasets via PSDs in the spectral domain could reduce mismatches and enhance pretraining. We then go beyond the direct intractable minimization optimization and innovatively reformulate it as a principled \textit{harmonization} approach. Specifically, we propose \textit{Harmonizer}, a module that reshapes spectral structures and implicitly harmonizing PSDs across datasets, which theoretically corresponds to a shared reparameterization of second-order temporal correlations. Our theoretical analysis further reveals token interactions with Harmonizer can be efficiently mediated by a compact set of resonators, motivating a \textit{HarmonicAttention} design that performs self-attention in a low-dimensional interaction space. Then, we propose \textit{Olivia}, a novel time series foundation model built upon these harmonization mechanisms. Extensive experiments on two large-scale benchmarks (TSLib and GIFT-Eval) and extra 6 datasets from GluonTS, demonstrate Olivia consistently achieves state-of-the-art performance under zero-shot, few-shot, and full-shot forecasting scenarios.

Reinforcement Learning · Batch/Offline

Yisen Zhao, Peixi Peng, Xinyu Hu, Cong Li, Zhan Su, Zhuojian Li

Offline meta-reinforcement learning requires agents to generalize to unseen tasks from fixed datasets, yet existing sequence-based and MoE-based methods rely on implicit or token-level routing signals that fail to capture task-level structure. We propose the **Task-Guided Router (TGR)**, a structured expert-routing framework that explicitly models inter-task relationships via multi-view task representations that combine semantic descriptors, behavioral summaries, and latent dynamics features. Using structure-guided routing, TGR assigns experts based on global task compatibility rather than local trajectory fragments, enabling stable specialization and effective knowledge transfer across tasks.Extensive experiments on continuous-control benchmarks demonstrate that TGR consistently outperforms state-of-the-art offline meta-RL methods in few-shot generalization, particularly under sparse data and heterogeneous dynamics. Our results highlight the importance of task-level priors for robust offline meta-reinforcement learning.

Applications · Computer Vision

Qishen Yin, Tanghui Jia, Peng Jin, Hao Li, Juntong Wu, Guanlin Lu, Li Yuan

Does \emph{Comprehending the main idea of a 2-hour movie} and \emph{Counting the birds appearing in a 15-second clip} really warrant the same video processing pipeline? We present Task-Aware Mechanism (TAM), a hybrid-gated Mixture-of-Experts (MoE) vision tower that adapts frame count and resolution to the user query and video length. TAM introduces a compact 0.1B text-only \emph{Inductor} trained on our TA-116K dataset to infer task types, enabling task-aware visual budgeting and routing: a soft-gated MoE vision encoder for stability, and hard-gated resolution-specific projectors/pipelines for efficient specialization. Built on Qwen2-7B, TallVA-8B-A7B achieves state-of-the-art performance among models with comparable LLMs across diverse video benchmarks and remains competitive against stronger-LLM baselines, showing that task-aware visual budgeting makes video understanding more holistic. The code is included in the supplementary material.

Applications · Neuroscience, Cognitive Science

Ganxi Xu, Zhao-Rong Lai, Yuting Tang, Yonghao Song, Guoxu Zhou, Boyu Wang, Jian Zhu, Jinyi Long

Visual prostheses hold great promise for restoring vision in blind individuals. While researchers have successfully utilized M/EEG signals to evoke visual perceptions during the brain decoding stage of visual prostheses, the complementary process of converting images into M/EEG signals in the brain encoding stage remains largely unexplored, hindering the formation of a complete functional pipeline. In this work, we present a novel image-to-brain signal framework that generates M/EEG from images by leveraging the diffusion transformer architecture enhanced with cross-attention mechanisms. Specifically, we employ a diffusion transformer (DiT) architecture based on denoising diffusion implicit models (DDIM) to achieve brain signal generation. To realize the goal of image-to-brain signal conversion, we use cross-attention mechanisms to align brain signal embeddings with CLIP image embeddings. Moreover, we leverage large language models (LLMs) to generate image captions, and concatenate the resulting CLIP text embeddings with CLIP image embeddings to form unified embeddings for cross-attention alignment, enabling our model to capture core semantic information. Furthermore, we introduce a learnable spatio-temporal position encoding that combines brain region embeddings with temporal embeddings to capture both spatial and temporal characteristics of brain signals. We evaluate the framework on two multimodal benchmark datasets (THINGS-EEG2 and THINGS-MEG) and demonstrate that it generates biologically plausible brain signals.

General Machine Learning · Unsupervised and Semi-supervised Learning

Xiaoyu Wang, Zhuoming Li, Bo Han, Hui LIU, Junhui Hou, Yuheng Jia

Single-Positive Multi-Label Learning (SPML) studies learning from incomplete supervision, where each instance is annotated with only one positive label despite potentially belonging to multiple categories. While existing methods assume the annotated labels are randomly distributed, real-world annotations are often biased toward the most salient category. We formalize this realistic scenario as Salient Single-Positive Multi-Label Learning (SalSPML). This salient annotation bias poses a challenge to conventional SPML methods, as the missing labels often correspond to less salient and harder-to-recognize categories. Fortunately, we find that salient annotations are typically more representative and informative. Motivated by this insight, we propose Prototype-Guided Rejection for Salient Annotation (PiSA), which constructs reliable class-wise prototypes from salient labels and leverages them to guide embedding learning for non-salient labels recognition. We theoretically demonstrate that SalSPML is harder than Random SPML due to irreducible annotation bias, and under SalSPML, more accurate prototypes facilitate false-negative label detection. Experiments on multiple benchmarks, together with two newly constructed real-world SalSPML datasets, demonstrate that PiSA consistently outperforms existing methods, achieving an average mAP improvement of 3.16\%. Our code is available in the supplementary materials.

General Machine Learning · Evaluation

Nicolas Salvy, Hugues Talbot, Thirion Bertrand

Generative model evaluation commonly relies on high-dimensional embedding spaces to compute distances between samples. We show that dataset representations in these spaces are affected by the hubness phenomenon, which distorts nearest neighbor relationships and biases distance-based metrics. Building on the classical Iterative Contextual Dissimilarity Measure (ICDM), we introduce Generative ICDM (GICDM), a method to correct neighborhood estimation for both real and generated data. We introduce a multi-scale extension to improve empirical behavior. Extensive experiments on synthetic and real benchmarks demonstrate that GICDM resolves hubness-induced failures, restores reliable metric behavior, and improves alignment with human judgment.

Applications · Time Series

Zhongzheng Qiao, SHENG PAN, Anni Wang, Viktoriya Zhukova, Yong Liu, Xudong Jiang, Qingsong Wen, Mingsheng Long, Ming Jin, Chenghao Liu

Time series foundation models (TSFMs) are revolutionizing the forecasting landscape from specific dataset modeling to generalizable task evaluation. However, we contend that existing benchmarks exhibit common limitations in four dimensions: constrained data composition dominated by reused legacy sources, compromised data integrity lacking rigorous quality assurance, misaligned task formulations detached from real-world contexts, and rigid analysis perspectives that obscure generalizable insights. To bridge these gaps, we introduce **TIME**, a next-generation task-centric benchmark comprising 50 fresh datasets and 98 forecasting tasks, tailored for strict zero-shot TSFM evaluation free from data leakage. Integrating large language models and human expertise, we establish a rigorous human-in-the-loop benchmark construction pipeline to ensure high data integrity and redefine task formulation by aligning forecasting configurations with real-world operational requirements and variate predictability. Furthermore, we propose a novel pattern-level evaluation perspective that moves beyond traditional dataset-level evaluations based on static meta labels. By leveraging structural time series features to characterize intrinsic temporal properties, this approach offers generalizable insights into model capabilities across diverse patterns. We evaluate 12 representative TSFMs and establish a multi-granular leaderboard to facilitate in-depth analysis and visualized inspection.

Deep Learning · Generative Models and Autoencoders

Donggyu Lee, Taekyung Lee, Jaewoong Choi

We investigate unpaired image inverse problems, a challenging setting where only independent, non-paired sets of noisy measurements and clean target signals are available for training. We propose a novel inverse problem solver based on Unbalanced Optimal Transport, called ***Unbalanced Optimal Transport Map for Inverse Problems (UOTIP)***. Our method formulates the reconstruction task—predicting clean target signals from noisy measurements—as learning a UOT Map from noisy measurement distribution to clean signal distribution by incorporating a likelihood-based cost function. By relaxing the exact marginal constraint, the UOT framework provides key advantages to our model: robustness to multi-level observation noise, adaptability to class imbalance between noisy and clean datasets, and generalizability to diverse noise-type scenarios. Furthermore, we theoretically demonstrate that incorporating a quadratic cost term ensures the existence and uniqueness of the transport map by satisfying the twist condition, even for ill-posed inverse problems. Our experiments demonstrate that UOTIP achieves state-of-the-art performance on unpaired image inverse problem benchmarks, across linear and nonlinear inverse problems.

Social Aspects · Safety

Sudarshan Regmi

The ability of the deep learning model to recognize when a sample falls outside its learned distribution is critical for safe and reliable deployment. Recent state-of-the-art out-of-distribution (OOD) detection methods leverage activation shaping to improve the separation between in-distribution (ID) and OOD inputs. These approaches resort to sample-specific scaling but apply a static percentile threshold across all samples regardless of their nature. In this work, we propose AdaSCALE, an adaptive scaling procedure that dynamically adjusts the percentile threshold based on a sample's estimated OOD likelihood. This estimation leverages our key observation that OOD samples exhibit significantly more pronounced activation shifts at high-magnitude activations under minor perturbation compared to ID samples. AdaSCALE enables stronger scaling for likely ID samples and weaker scaling for likely OOD samples, creating highly separable energy scores. Our approach achieves state-of-the-art OOD detection performance, outperforming the latest rival OptFS by **14.94%** in near-OOD and **21.67%** in far-OOD datasets in average FPR@95 metric in the ImageNet-1k benchmark across eight diverse architectures.

Applications · Everything Else

Shirley Wu, Evelyn Choi, Arpandeep Khatua, Zhanghan Wang, Joy He-Yueya, Cyril Weerasooriya, Wei Wei, Diyi Yang, Jure Leskovec, James Zou

Large Language Models (LLMs) are increasingly used to simulate how specific users respond to any context, enabling more user-centric applications that rely on user feedback. However, existing user simulators mostly imitate surface-level patterns and language styles, which fails to reflect the underlying state of real users (e.g., beliefs, emotions). To address these limitations, we propose a novel training framework, HumanLM, which builds user simulators that accurately reflect real users. Our key insight is, in addition to generating responses, we generate natural-language latent states that align with the ground truth responses through reinforcement learning. These latent states correspond to a set of state dimensions which psychologically lead to how real users respond. HumanLM further synthesizes these aligned latent states into responses that accurately represent real users. For extensive evaluation, we develop Humanual, a comprehensive benchmark on simulating real users based on public data. Humanual consists of six large-scale datasets with 26k users and 216k responses in total. It spans diverse tasks such as generating user responses to daily life issues, political blogs, and chat sessions with LLM assistants. Across the datasets, HumanLM significantly outperforms the best alternative approaches by an average relative improvement of 16.3% on alignment score from an LLM judge. In a real-time simulation study with 37 participants, HumanLM achieves the highest scores on similarity with real user responses and human-likeness.

Applications · Health / Medicine

Jun Li, ZIWEI QIN

Semi-supervised learning has become a dominant paradigm for reducing annotation costs. However, we argue that the current progress is clouded by a dual problem of overconfidence. Algorithmically, prevailing pseudo-labeling frameworks often conflate prediction confidence with uncertainty, leading to severe confirmation bias and poorly calibrated models. Methodologically, since multiple benchmark datasets lack validation sets, some studies repurpose test sets as validation sets, leading to inflated results. Subsequent methods, compelled to employ the same strategy to surpass reported SOTA, trigger an arms race of overfitting. Exciting numerical gains may reflect test overfitting rather than genuine progress. Thus, we propose TCSeg, a tri-space calibrated segmentation framework founded on a principled dual-axis reliability assessment engine. It explicitly decouples confidence from uncertainty and uses this signal to detect and correct confirmation bias across feature, probability, and image spaces in a collaborative manner. Across three benchmarks, TCSeg delivers consistently strong performance at both the best and final checkpoints, and we report results under a multi-run protocol to reset the benchmark with a more realistic perspective. Core code is available: https://github.com/BubbleDirk/temporary_anonymous_tcseg.

Applications · Computer Vision

Yu Li, Zhengran Shen, Yachun Mi, Puchao Zhou, Shaohui Liu

Leveraging Large Vision-Language Models like CLIP has recently set new benchmarks for No-Reference Image Quality Assessment (NR-IQA). However, the contrastive pretraining of CLIP inherently prioritizes semantic invariance, which often suppresses subtle perceptual signals, a phenomenon we term perceptual submergence. Furthermore, standard preprocessing techniques (e.g., cropping and interpolation) further exacerbate the loss of critical high-frequency quality cues. In this paper, we propose the Cross-modal Perception Alignment Adapter (CMPA), a manifold-aware framework designed to disentangle perceptual distortions from dominant semantics. CMPA introduces a Perception-Sensitive Feature Extractor (PFE) that projects CLIP features into a compact, low-dimensional subspace, explicitly magnifying distortion-induced off-manifold deviations. Subsequently, a Cross-Modal Perception Alignment Injector (PAI) aligns these features with quality-aware text anchors and re-injects them into the backbone. To ensure input fidelity, we also devise a Residual-enhanced Perceptual Downscaling strategy that adaptively compensates for resolution-induced information loss using Just Noticeable Difference (JND) guided frequency re-injection. Extensive evaluations on several benchmark datasets demonstrate that our approach significantly outperforms state-of-the-art methods, effectively recovering the perceptual signals submerged in semantic-dense representations.

Applications · Chemistry, Physics, and Earth Sciences

Xudong Ling, Lichaorong, Huang Tianxi, Qian Dong, Guiduo Duan

Short-term precipitation nowcasting is inherently under-constrained due to limited historical observation windows: identical observations can lead to multiple plausible future trajectories, especially for extreme events. Existing generative methods rely solely on visual features and lack explicit constraints on precipitation motion semantics, resulting in ambiguous dynamics, blurred details, and unstable predictions. We propose LangPrecip, the first language-guided precipitation nowcasting framework, and contribute LangPrecip-160K, a large-scale radar-text paired dataset with 160K annotated sequences. LangPrecip addresses the under-constrained challenge by leveraging natural-language motion descriptions as explicit semantic constraints to reduce motion ambiguity and introducing a dual-path wavelet consistency unfolding decoder that enforces physical data fidelity during latent-to-pixel reconstruction. By reformulating nowcasting as semantically constrained trajectory generation under the Rectified Flow paradigm with model-based decoder optimization, LangPrecip produces sharper and more physically consistent forecasts. Experiments on Swedish and MRMS benchmarks demonstrate substantial improvements over state-of-the-art vision-only methods, achieving over 60\% and 19\% relative gains in heavy-rainfall CSI at 80-minute lead time with enhanced spatial detail preservation.

Deep Learning · Generative Models and Autoencoders

Eitan Kosman, Gabriele Serussi, Chaim Baskin

Modality translation is inherently under-constrained, as multiple cross-modal mappings may yield the same marginals. Recent work has shown that diffusion bridges are effective for this task. However, most existing approaches rely on fully paired datasets, thereby imposing a single data-driven constraint. We propose a diffusion-bridge framework that characterizes the space of admissible solutions and restricts it via alignment constraints, treating paired supervision as an optional heuristic rather than a prerequisite. We validate our method on synthetic and real modality translation benchmarks across unpaired, semi-paired, and paired regimes, showing consistent performance across supervision levels. Notably, it achieves near fully-paired quality with a substantial relaxation in pairing requirements, and remaining applicable in the unpaired regime. These results highlight diffusion bridges as a flexible foundation for modality translation beyond fully paired data.

Applications · Neuroscience, Cognitive Science

Mo Wang, Wenhao Ye, Junfeng Xia, Junxiang Zhang, Xuanye Pan, Minghao Xu, Haotian Deng, Hongkai Wen, Quanying Liu

Self-supervised fMRI foundation models have shown promising transfer performance, yet most rely on predefined region-level parcellations that discard fine-grained voxel information and introduce atlas-dependent biases. We propose Omni-fMRI, an atlas-free foundation model that operates directly on voxel-level signals. To enable scalable pretraining on 49,497 fMRI sessions across nine datasets, Omni-fMRI introduces a dynamic patching mechanism that substantially reduces computational cost while preserving informative spatial structure. To support reproducibility and fair comparison, we establish a comprehensive benchmark suite spanning 11 datasets and a diverse set of resting-state and task-based fMRI tasks. Experimental results demonstrate that Omni-fMRI consistently outperforms existing foundation models, providing a scalable and reproducible framework for atlas-free brain representation learning. Code is available.