论文检索

Applications · Chemistry, Physics, and Earth Sciences

Jun Chen, Yan Fang, Minghui Qiu, Yueran Qiu, Lin CHEN, Shuxin Zhong, Yu ZHANG, Kaishun WU

Nowcasting forms the first line of defense against rapidly evolving weather hazards, where even minutes of delay can lead to severe societal impacts. However, existing systems predominantly extrapolate 2D radar reflectivity, which struggles under rapid intensification regimes. We introduce \N, a multi-scale modeling framework that enables coherent reconstruction of the three-dimensional evolution of convective systems while explicitly conditioning on the ambient environment. \N\ integrates multi-source observations—including radar, satellite, and station—with reanalysis fields through two components: (i) \texttt{\ComponentA} that explicitly disentangles convective system state form vertical thermodynamic coupling and large-scale environmental forcing; (ii) \texttt{\ComponentB} that predicts future echos by adaptively aggregating cross-layer interactions conditioned on evolving environmental conditions. To support comprehensive evaluation, we build a new benchmark \texttt{StormBench} that integrates observational and reanalysis data across regions. On this benchmark,~\N\ consistently achieves the best performance, reducing MAE by 12.4\% and improving the mCSI by 34.0\%. \textit{Dataset and code will be released after the review process}.

Gül Sena Altıntaş, Malikeh Ehghaghi, Brian Lester, Fengyuan Liu, Wanru Zhao, Marco Ciccone, Colin Raffel

Tokenizers provide the fundamental basis through which text is represented and processed by language models (LMs). Despite the importance of tokenization, its role in LM performance and behavior is poorly understood due to the challenge of measuring the impact of tokenization in isolation. To address this need, we present TokSuite, a collection of models and a benchmark that supports research into tokenization's influence on LMs. Specifically, we release fourteen pre-trained models that use different tokenizers but are otherwise identical, using the same architecture, dataset, training budget, and initialization. We also release a multilingual robustness benchmark that measures model performance under real-world perturbations in English, Chinese, Farsi, Italian, and Turkish, curated by native annotators. Together, TokSuite allows robust decoupling of the influence of a model's tokenizer, supporting a series of novel findings that elucidate the respective benefits and shortcomings of a wide range of popular tokenizers.

General Machine Learning · Representation Learning

yan wubin, Wei Ma, Shixiang Wan, Dongchen Li, Shaoshun Kang, Qing Yang, Dongliang Xu

Connectionist models and symbolic models have long embodied two divergent paradigms: the former excel at differentiable representation learning yet struggle with transparency, while the latter deliver explicit rule-based reasoning but resist gradient-based optimization. We introduce Arboreal Neural Networks (ArbNN), a neural–symbolic framework that unifies these paradigms both computationally and conceptually. At the design level, ArbNN departs fundamentally from prior neuralized-tree models through a depth-aware routing mechanism and a topology-informed softmax aggregation, which together enable one-shot full-path gradient propagation and consequently achieving rapid and well-conditioned optimization dynamics and high parallel inference efficiency. At the conceptual level, ArbNN reveals that decision-tree branching and self-attention routing are two realizations of the same conditional computation primitive. We prove a structural isomorphism between a decision tree and a single-query attention head, enabling a differentiable architecture that faithfully preserves symbolic decision logic. The defining property of ArbNN is Bidirectional Fidelity, ensuring that the neural module can be compiled from—and losslessly decompiled back into—a symbolic tree, yielding both ordering consistency in ranking behavior and explicit, auditable interpretability via reconstructed if–else rules. ArbNN further supports GBDT-based initialization, allowing it to inherit strong inductive biases and integrate seamlessly with existing production workflows. Empirically, ArbNN achieves state-of-the-art performance on various public tabular benchmarks and delivers consistent gains under temporal distribution shift in large-scale industrial credit-risk systems. To support realistic evaluation, we additionally contribute TabCredit, a feature-rich, temporally partitioned dataset built from millions of real-world loan applications. Together, these results demonstrate that ArbNN forms a unified, reversible, and practically deployable bridge between symbolic reasoning and neural computation for high-stakes tabular domains.

Applications · Robotics

Jung Min Lee, Dohyeok Lee, Seokhun Ju, Taehyun Cho, Jin Koo, Li Zhao, Sangwoo Hong, Jungwoo Lee

Learning *latent actions* from diverse human videos enables scaling robot learning beyond embodiment-specific robot datasets, and these latent actions have recently been used as pseudo-action labels for vision-language-action (VLA) model pretraining. To make VLA pretraining effective, latent actions should contain information about the underlying agent's actions despite the absence of ground-truth labels. We propose **M**ulti-**V**iew**P**oint **L**atent **A**ction **M**odel (**MVP-LAM**), which learns discrete latent actions that are highly informative about ground-truth actions from time-synchronized multi-view videos. MVP-LAM trains latent actions with a *cross-viewpoint reconstruction* objective, so that a latent action inferred from one view must explain the future in another view, reducing reliance on viewpoint-specific cues. On Bridge V2, MVP-LAM produces more action-centric latent actions, achieving higher mutual information with ground-truth actions and improved action prediction, including under out-of-distribution evaluation. Finally, pretraining VLAs with MVP-LAM latent actions improves downstream manipulation performance on the SIMPLER and LIBERO-Long benchmarks.

Social Aspects · Accountability, Transparency, and Interpretability

Millicent Li, Alberto Mario Ceballos Arroyo, Giordano Rogers, Naomi Saphra, Byron Wallace

Recent interpretability methods have proposed to translate LLM internal representations into natural language descriptions using a second verbalizer LLM. This is intended to illuminate how the target model represents and operates on inputs. But do such activation verbalization approaches actually provide privileged knowledge about the internal workings of the target model, or do they merely convey information about its inputs? We critically evaluate popular verbalization methods across datasets used in prior work and find that they can succeed at benchmarks without any access to target model internals, suggesting that these datasets may not be ideal for evaluating verbalization methods. We then run controlled experiments which reveal that verbalizations often reflect the parametric knowledge of the verbalizer LLM which generated them, rather than the knowledge of the target LLM whose activations are decoded. Taken together, our results indicate a need for targeted benchmarks and experimental controls to rigorously assess whether verbalization methods provide meaningful insights into the operations of LLMs.

Deep Learning · Large Language Models

Yang Song, Anoushka Vyas, Zirui Wei, Sina Pakazad, Henrik Ohlsson, Graham Neubig

In this paper, we present **NEMO**, a system that translates **N**atural-language descriptions of decision problems into formal **E**xecutable **M**athematical **O**ptimization implementations, operating collaboratively with users or autonomously. Existing approaches typically rely on specialized large language models (LLMs) or bespoke, task-specific agents. Such methods are often brittle, complex and frequently generating syntactically invalid or non-executable code. NEMO instead centers on remote interaction with autonomous coding agents (ACAs), treated as a first-class abstraction analogous to API-based interaction with LLMs. This design enables the construction of higher-level systems around ACAs that structure, consolidate, and iteratively refine task specifications. Because ACAs execute within sandboxed environments, code produced by NEMO is executable by construction, allowing automated validation and repair. Building on this, we introduce novel coordination patterns with and across ACAs, including asymmetric validation loops between independently generated optimizer and simulator implementations (serving as a high-level validation mechanism), external memory for experience reuse, and robustness enhancements via minimum Bayes risk (MBR) decoding and self-consistency. We evaluate NEMO on nine established optimization benchmarks. As depicted in Figure 1, it achieves state-of-the-art performance on the majority of tasks, with substantial margins on several datasets, demonstrating the power of execution-aware agentic architectures for automated optimization modeling.

General Machine Learning · Clustering

Qian Guo, Gaohui Zuo, Bingbing Jiang, Guangrui Fan, Zhihua Cui, Xinyan Liang, Jianjian Ding

Incomplete multi-view clustering (IMVC) aims to uncover shared clustering structures from heterogeneous views with partial observations. Recently, existing generative IMVC methods have made significant progress in this field; however, they still remain limited in two aspects. On the one hand, they rely on weak cross-view signals, resulting in unstable latent recovery when facing heterogeneous missing data. On the other hand, they overlook stable cross-view neighborhood structures, leading to weak structural constraint. To address these limitations, we propose neighborhood-conditioned diffusion for incomplete multi-view clustering (IMVC-NCD), which achieves robust latent completion. Our method learns compact view-specific latent representations and constructs a unified conditioning vector by aggregating stable local neighborhood structures from available views while encoding heterogeneous missingness states, providing reliable guidance for diffusion-based denoising. With neighborhood-level conditioning, IMVC-NCD produces semantically aligned and view-consistent latent representations that are well suited for clustering, even under high missing-view ratios. Extensive experiments on four benchmark datasets demonstrate the effectiveness and robustness of our method compared with state-of-the-art IMVC approaches.

Reinforcement Learning · Batch/Offline

Yantian Wang, Wenhao Li, Bo Jin

Optimizing maintenance strategies for large-scale infrastructure is a critical sequential decision-making problem, exemplified by the high-stakes domain of bridge management. While Reinforcement Learning (RL) offers a theoretical framework for such problems, practical deployment necessitates offline constrained RL—learning policies solely from static historical datasets under rigid budgetary limits without dangerous on-policy exploration.However, current research is hindered by benchmarks that fail to capture the confluence of distributional shift and hard constraints typical of real-world assets. We introduce InfraRL, a high-fidelity benchmark that uses bridge maintenance as a rigorous testbed for general infrastructure asset management challenges.Constructed from the U.S. National Bridge Inventory, InfraRL defines a rigorous offline task for optimizing maintenance strategies under hard budgetary constraints. We benchmark a diverse suite of baselines, ranging from industry-standard heuristics to SOTA single-agent and multi-agent offline RL algorithms. Through a comprehensive evaluation protocol, we analyze performance across structural utility, constraint adherence, and behavioral fidelity, revealing critical trade-offs between safety and long-term efficiency. Our code and data are available at https://anonymous.4open.science/r/ICML-6656

Deep Learning · Large Language Models

Yu Wang, Yijian Liu, Liheng Ji, Han Luo, Wenjie Li, Xiaofei Zhou, Chiyun Feng, Puji Wang, Yuhan Cao, Geyuan Zhang 等

Large language models (LLMs) have demonstrated remarkable capabilities across a variety of domains. However, their applications in cryptography, which serve as a foundational pillar of cybersecurity, remain largely unexplored. To address this gap, we build \textbf{AICrypto}, a comprehensive benchmark designed to evaluate the cryptography capabilities of LLMs. The benchmark comprises 135 multiple-choice questions, 150 capture-the-flag challenges, and 30 proof problems, covering a broad range of skills from knowledge memorization to vulnerability exploitation and formal reasoning. All tasks are carefully reviewed or constructed by cryptography experts to improve correctness and rigor. For each proof problem, we provide detailed scoring rubrics and reference solutions that enable automated grading, achieving high correlation with human expert evaluations. We introduce strong human expert performance baselines for comparison across all task types. Our evaluation of 17 leading LLMs reveals that state-of-the-art models match or even surpass human experts in memorizing cryptographic concepts, exploiting common vulnerabilities, and routine proofs. However, our analysis reveals that they still lack a deep understanding of abstract mathematical concepts and struggle with tasks that require multi-step reasoning and dynamic analysis. We hope this work could provide insights for future research on LLMs in cryptographic applications. Our code and dataset are available at https://anonymous.4open.science/r/aicrypto-CE6E/.

General Machine Learning · Evaluation

YongKyung Oh

State-of-the-Art (SOTA) claims pervade Artificial Intelligence (AI) and Machine Learning (ML) research. These claims rest on benchmark evaluations, where models are ranked by aggregate scores across tasks. Public leaderboards are the most visible instance, but the same structure appears in paper tables throughout the literature. However, such minimal evidence often cannot support these strong claims. We identify a widespread claim-evidence gap in AI benchmarking. Claiming SOTA implies robust superiority. It suggests that a model significantly outperforms alternatives across most tasks. However, a marginal improvement in the mean score merely indicates a top average rank rather than true superiority. Analyzing ten cross-domain benchmarks from public leaderboards, we found that in more than half of top-model comparisons, at least one commonly assumed property of superiority does not hold. These properties include meaningful effect size, consistency across tasks, or robustness to dataset removal. Instead, aggregate gains are frequently driven by outlier datasets. This fragility persists even in benchmarks with many tasks. We argue that claim language should reflect the strength of the underlying evidence. This requires no additional experiments, only honest reporting of what results actually show.

General Machine Learning · Evaluation

Anna Genevaux, Simon Frieder

This position paper argues that documentation is infrastructure for reproducible geometry reasoning: a benchmark for formal geometry problems to test AI systems is not usable in research unless its documented vocabulary is matched by executable, versioned behavior and minimal runnable examples. We use JGEX (as implemented by Newclid) as a case study of how documentation--implementation gaps and missing examples can silently constrain expressivity, fragment tool interoperability, and bias benchmark construction. To make our point, we introduce "A JGEX Dataset", a curated collection of $78$ Euclidean geometry problems with (i) original natural-language statements and sources, (ii) a JGEX-oriented rewrite that makes formalization steps explicit, (iii) executable JGEX code validated under a pinned solver version, and (iv) rich metadata. To make the target language auditable, we also provide a predicate-level support matrix for the $33$ documented predicates, generated from minimal test instances, and categorize predicates as supported, unsupported, or unstable due to missing accessible examples. Finally, we release validation scripts and a concise tutorial with worked walk-throughs. Our broader claim is that benchmark authors, tool maintainers, and reviewers should treat language documentation and conformance evidence as first-class artifacts—on par with datasets and evaluation code—if cross-tool, cross-version reproducibility is the goal.

Social Aspects · Safety

Keonwoo Kim, Hyeseon Ko, Hyejeong Jo, Sewon Kim, Yera Choi, JaeDeok Lee, Heeyoung Kwak, Yunwook Sung, Haanju Yoo

As medical large language models become increasingly involved in clinical actions, public benchmarks are often treated as proxies of deployment-readiness. However, this reliance creates a false sense of security because public scores are often based on data the models have already seen. We call this the Open Benchmark Paradox: making evaluation data public for research progress also makes data contamination inevitable, ruining its value as a reliable safety signal. This paradox induces three structural failures: (1) hidden contamination, where it is impossible to prove evaluation independence; (2) outdated standards, where static datasets fail to track evolving medical guidelines; and (3) jurisdictional divergence, where global averaging ignores local legal and ethical standards. To validate these risks, we audited frontier models using recent medical exam data, which confirmed a high probability of data contamination. To resolve such integrity issues in medical evaluation, we propose Sovereign Medical Evaluation (SME). Instead of public leaderboards, SME establishes a national infrastructure where health authorities manage private, isolated evaluation pipelines. Within this secure system, evaluations are automatically updated using live medical data and legal changes, ensuring they remain current and strictly separated from model training. SME provides the essential transition to a controlled, auditable, and legally grounded safety gate for medical AI.

Reinforcement Learning · Batch/Offline

Anthony Kobanda, Waris Radji, Odalric-Ambrym Maillard, Rémy Portelas

Goal-Conditioned Reinforcement Learning aims to design agents that can reach specified goals, notably from previously collected trajectories in the offline setting. In this context, graph-based approaches have been proposed to mitigate compounding value-estimation errors in long-horizon navigation tasks. However, existing approaches typically rely on dense keypoint coverage of the dataset support, resulting in computationally expensive planning. Moreover, they lack explicit mechanisms to adapt to topological changes (e.g., new obstacles), hindering deployment in live applications such as video game environments. To address these two shortcomings, we introduce Adaptive Quasimetric Mapping (AQM), an offline framework leveraging a “time-to-reach” quasimetric learned from the available data. Crucially, it builds a sparse cover of the dataset support, as a greedy approximation to a dominating set problem. At test-time, the resulting graph is carefully pruned by comparing the observed edge traversal time against a time-to-reach budget derived from the quasimetric, thus enabling zero-shot replanning. Empirically, we evaluate AQM on navigation tasks ranging from a classical to a video-game-like benchmark evaluating adaptation across tasks. We show that AQM achieves competitive performance while requiring up to 100× fewer keypoints than prior approaches, hence demonstrating the relevance of topological abstraction for goal-conditioned navigation.

General Machine Learning · Evaluation

Zachary Berger, Daniel Prakah-Asante, John Guttag, Collin Stultz

This position paper argues that current benchmarking practice in 12-lead ECG representation learning must be fixed to ensure progress is reliable and aligned with clinically meaningful objectives. The field has largely converged on three public multi-label benchmarks (PTB-XL, CPSC2018, CSN) dominated by arrhythmia and waveform-morphology labels, even though the ECG is known to encode substantially broader clinical information. We argue that downstream evaluation should expand to include an assessment of structural heart disease and patient-level forecasting, in addition to other evolving ECG-related endpoints, as relevant clinical targets. Next, we outline evaluation best practices for multi-label, imbalanced settings, and show that when they are applied, the literature's current conclusion about which representations perform best is altered. Furthermore, we demonstrate the surprising result that a randomly initialized encoder with linear evaluation matches state-of-the-art pre-training on many tasks. This motivates the use of a random encoder as a reasonable baseline model. We substantiate our observations with an empirical evaluation of three representative ECG pre-training approaches across six evaluation settings: the three standard benchmarks, a structural disease dataset, hemodynamic inference, and patient forecasting.

Deep Learning · Large Language Models

Zongqi Wang, Rui Wang, Yuchuan Wu, Yiyao Yu, Pinyi Zhang, Shaoning Sun, Yujiu Yang, Yongbin Li

Reinforcement Learning with Verifiable Reward (RLVR) on preference data has become the mainstream approach for training Generative Reward Models (GRMs). Typically, GRMs generate reasoning chains ending with critiques and preference labels, with RLVR using label correctness as the training reward. However, we demonstrate that such binary classification tasks make GRMs susceptible to guessing correct outcomes without sound critiques, introducing noise into the reward signal and impairing learning effectiveness. To address this, we propose Reward Modeling from Natural Language Human Feedback (RM-NLHF), which leverages natural language feedback to obtain process reward signals. Specifically, we compute the similarity between GRM-generated and human critiques as the process reward, providing more accurate signals than outcome-only supervision. Considering that human critiques are difficult to scale, we introduce MetaRM which learns to predict process reward from datasets with human critiques and generalizes to data without them. Experiments on multiple benchmarks demonstrate that RM-NLHF consistently outperforms state-of-the-art models trained with outcome reward, confirming the superiority of natural language over binary feedback.

Social Aspects · Safety

Janis Keuper, Margret Keuper

The wide availability and low usability barrier of modern image generation models has triggered the reasonable fear of criminal misconduct and negative social implications. The machine learning community has been engaging this problem with an extensive series of publications proposing algorithmic solutions for the detection of "fake'', e.g. entirely generated or partially manipulated images. While there is undoubtedly some progress towards technical solutions of the problem, we argue that current and prior work is focusing too much on generative algorithms and "fake'' data-samples, neglecting a clear definition and data collection of "real'' images. The fundamental question *"what is a real image?''* might appear to be quite philosophical, but our analysis shows that the development and evaluation of basically all current "fake''-detection methods is relying on only a few, quite old low-resolution datasets of "real'' images like *ImageNet*. However, the technology for the acquisition of "real'' images, aka taking photos, has drastically evolved over the last decade: Today, over 90% of all photographs are produced by smartphones which typically use algorithms to compute an image from multiple inputs (over time) from multiple sensors. Based on the fact that these image formation algorithms are typically neural network architectures which are closely related to "fake''-image generators, we state the position that today, **we need to re-think the concept of "real'' images**. The purpose of this position paper is to raise the awareness of the current shortcomings in this active field of research and to trigger an open discussion whether the detection of ``fake'' images is a sound objective at all. At the very least, we need a clear technical definition of "real'' images and new benchmark datasets.

Applications · Time Series

Md Asif Bin Syed, Md Younus Ahamed, Azmine Toushik Wasi

Time series foundation models (TSFMs) have demonstrated strong performance on established benchmarks such as GIFT-Eval, Monash, and TSFM-Bench. However, these benchmarks pool datasets from many domains with uneven representation, which can obscure performance within specific application areas such as healthcare, finance, nature, retail, and transport. The necessity for domain-specific evaluation arises from the inherent structural diversity of time series data: clinical records often feature irregular sampling and informative missingness; financial sequences are characterized by high noise and stochastic trajectories; and environmental data, such as energy and weather, are governed by deterministic physical laws and strong seasonal hierarchies. Motivated by this heterogeneity, **we argue that TSFMs require explicit domain-specific benchmarks** so practitioners can reliably assess a model's utility within their own application area. This is because cross-domain differences in data generation, sampling irregularity, and nonstationarity under concept drift fundamentally shape forecasting difficulty and failure modes. As a result, strong performance on aggregated leaderboards may not translate to reliable deployment within a specific domain. To test this, we evaluated seven TSFMs across 72 datasets from six domains (healthcare, finance, energy, nature, transport, and retail) and found substantial cross-domain variability. These findings confirm that global benchmark scores can be misleading and that domain-aware evaluations are essential for trustworthy TSFM selection.

Deep Learning · Sequential Models, Time series

Md Mahmuddun Nabi Murad, Yasin Yilmaz

Early and accurate detection of anomalies in time-series data is critical due to the substantial risks associated with false or missed detections. While MLP-based mixer models have shown promise in time-series analysis, they do not maintain temporal causality during data processing. Moreover, real-world multivariate time series often contain numerous channels with diverse inter-channel correlations. Spurious correlations in the reconstructed time series lead to noisy representations, resulting in inaccurate anomaly detection. In addition, anomaly scoring methods that ignore temporal continuity can mislead sequential detection. To address these challenges, we propose a cluster-aware causal mixer for multivariate time-series anomaly detection. Channels are grouped into clusters based on their correlations, and each cluster is embedded through a dedicated embedding layer. A causal mixer is introduced to integrate information while maintaining temporal causality. We further develop a sequential anomaly-scoring method that accumulates evidence over time and refines anomaly boundaries. Our proposed model operates in an online fashion, making it suitable for real-time time-series anomaly detection. Experimental evaluations across six public benchmark datasets demonstrate that the proposed approach consistently achieves superior performance.

Reinforcement Learning · Deep RL

Lucas Fagan, Michele Tarquini, Ali Shehper, Maksymilian Manko, Angus Gruen, Coco Huang, Giorgi Butbaia, Davide Passaro, Sergei Gukov

Mathematical search problems present a unique challenge for Reinforcement Learning (RL) due to vast search spaces and sparse rewards. In previous works, the Andrews-Curtis (AC) conjecture was established as an illustrative example of such problems. In this work, we identify a critical structural barrier in the AC landscape: a "Two Hump" distribution, where problem instances are either trivially solvable or effectively impossible, with a scarcity of intermediate "hard-but-solvable" instances required for effective learning. We tackle this challenge through two primary avenues: novel data generation techniques to populate the difficulty gap, and significant algorithmic enhancements including the introduction of supermoves and Transformer-based architectures. We demonstrate substantial performance improvements over previous baselines, and release new comprehensive benchmark datasets including **AC-19** (125,192 AC-trivial presentations of varying difficulty with length at most 19) and **AC-1M** (1,136,154 hard AC-trivial presentations of length at most 30), the first large-scale, publicly available datasets of this kind.

Applications · Chemistry, Physics, and Earth Sciences

Sihan Wang, Wenjie Du, Yang Wang

In the AI4Chemistry scenario, utilizing heterogeneous data at different fidelity levels is a common and core issue. High-fidelity data is accurate but scarce, while low-fidelity data is abundant but biased. Traditional multi-fidelity methods typically identify cross-fidelity biases based on paired samples under different fidelity labels. However, due to the mismatch in dataset input distribution and the complexity of the biases themselves, these methods are difficult to implement in real-world scientific environments. To address this, we propose a trusted information subset decomposition framework that can efficiently utilize multi-fidelity data without requiring paired samples. Multi-fidelity label supervision is decomposed into three complementary subsets: a trusted information subset based on the absolute value of high-fidelity labels; a trusted subset that captures the reliability of the high-fidelity and medium-fidelity label intervals through adaptive constraints; and an ordered trusted subset representing the numerical relationships within the same fidelity level. These subsets are then integrated into a unified end-to-end model, enabling the reasonable utilization of medium- and low-fidelity information. Extensive experiments on various molecular and material property benchmarks demonstrate that our method consistently outperforms state-of-the-art multifidelity and singlefidelity baseline methods, and exhibits good robustness under real-world unpaired multifidelity conditions.