论文检索

Deep Learning · Sequential Models, Time series

Ruben Solozabal, Velibor Bojkovic, Hilal AlQuabeh, Klea Ziu, Kentaro Inui, Martin Takac

State-space models (SSMs) have emerged as a powerful foundation for long-range sequence modeling, with the HiPPO framework showing that continuous-time projection operators can be used to derive stable, memory-efficient dynamical systems that encode the past history of the input signal. However, existing projection-based SSMs often rely on polynomial bases with global temporal support, whose inductive biases are poorly matched to signals exhibiting localized or transient structure. In this work, we introduce \emph{WaveSSM}, a collection of SSMs constructed over wavelet frames. Our key observation is that wavelet frames yield a localized support on the temporal dimension, useful for tasks requiring precise localization. Empirically, we show that on equal conditions, \textit{WaveSSM} outperforms orthogonal counterparts as S4 on real-world datasets with transient dynamics, including physiological signals on the PTB-XL dataset and raw audio on Speech Commands.

Social Aspects · Accountability, Transparency, and Interpretability

Zhuonan Yang, Jacob Xiaochen Li, Francisco Velez, Eric Todd, David Bau, Michael L. Littman, Stephen Bach, Ellie Pavlick

One of the most common complaints about large language models (LLMs) is their prompt sensitivity---i.e., the fact that their ability to perform a task or provide a correct answer to a question can depend unpredictably on the way the question is posed. We investigate this variation by comparing two very different but commonly-used styles of prompting: those using $\textit{instructions}$, which describe the task in natural language, and those using in-context $\textit{examples}$, which provide few-shot demonstration pairs to illustrate the task. We find that, despite large variation in performance as a function of the prompt, all the prompts we study engage the same common underlying mechanisms. Specifically, we identify task-specific heads that are interpretable in vocabulary space--- which we dub $\textit{lexical task heads}$---and show that these heads are shared across prompting styles and are essential to triggering subsequent answer production. We further find that behavioral variation between prompts can be explained by the degree to which these heads are activated, and that failures are at least sometimes due to competing task representations that dilute the signal of the target task. Our results together present an increasingly clear picture of how LLMs' internal representations can explain behavior that otherwise seems idiosyncratic to users and developers.

Social Aspects · Alignment

Tushita Jha, Rory Švarc, Mateusz Bagiński

The ML literature contains many distinct concepts falling under the heading of ‘AI alignment’. After noting three concepts of AI alignment and situating these ideals in the context of their corresponding research programs, we claim that realistic interventions may promote ‘AI alignment’ under one conception while being actively counterproductive from the perspective of others. We suggest that tensions between alignment ideals emerge due to differences in background threat-models, alongside differences in both methodological and normative orientations. In light of our analysis, researchers taking themselves to produce research aimed to further the goal of ‘AI alignment’ should do three things. First, they should distinguish between ‘AI alignment’ as a high-level ideal and the specific ‘alignment proxies’ used in empirical research. Second, they should use more granular concepts to identify the source in addition to the nature of possible AI harms/benefits. Third, they explicitly specify the non-technical background commitments motivating specific conceptions of ‘AI alignment’.

Applications · Everything Else

Tong Chen, Siyu Liu, Hongyi Zhong, liao zhang, Lixiang Wang, Xiwei Wu, Junchi Yan, Qinxiang Cao

Automatic loop invariant generation remains a challenging problem in program verification, particularly for memory-manipulating programs where shape invariants are required to characterize heap-allocated structures and memory layouts. While existing approaches succeed on numerical invariants, they achieve limited accuracy on shape invariants. We hypothesize that this stems from the need to reason about memory state evolution—information that remains implicit in source code. To address this, we ground LLM reasoning in symbolic execution traces that explicitly capture such transitions. We propose \textsc{Malice}, a two-stage framework incorporating these traces: (1) guided multi-step reasoning that predicts invariants via chain-of-thought analysis of traces, and (2) agentic iterative refinement that corrects candidates through verification tool feedback. Evaluated on LIG-MM+, a benchmark featuring common operations on typical memory structures, \textsc{Malice} substantially outperforms existing approaches.

Applications · Health / Medicine

Shizhao Yang, Yixin Wang, Kevin Lin

Deciphering how cells commit to future fates is essential for developing precision therapeutics that can reprogram stem cells or modulate immune functions. However, isolating these fate-determining signals in single-cell lineage tracing (scLT) remains challenging because differentiation programs are often confounded by unrelated processes like the cell cycle. To address this, we introduce Lineage-aware Contrastive Learning (LCL), a framework that treats inheritable lineage barcodes as a "natural" data augmentation to isolate subtle, lineage-specific signals. LCL utilizes a semi-supervised architecture to align unlabeled cells, facilitating the transfer of lineage structures to clinical datasets where explicit barcoding is unavailable. We demonstrate LCL’s utility by predicting future cell-type compositions from early-time points, effectively modeling longitudinal fate commitment from cross-sectional data. Benchmarking on hematopoietic and fibroblast systems shows that LCL significantly outperforms standard models like scVI, establishing contrastive learning as a scalable paradigm for understanding and potentially manipulating cellular differentiation.

Social Aspects · Safety

Jinjie Shen, Yaxiong Wang, Yujiao Wu, Lechao Cheng, Tianrui Hui, Nan Pu, Zhihui Li, Zhun Zhong

The rapid rise of generative AI has made multimodal fake news increasingly realistic and pervasive, posing severe threats to public trust and social stability. Existing detection methods rely heavily on manipulation-specific models and large-scale labeled data, resulting in poor generalization to emerging manipulation types. We observed that the essence of manipulated misinformation lies in its intrinsic conflicts, i.e., semantic or physical inconsistencies either across modalities or with common world knowledge. Inspired by this observation, we propose Conflict-{Oriented REasoning (CORE) framework, an effective paradigm that learns to endows multimodal large language models (MLLMs) with explicit conflict-capturing capability. To this end, CORE first constructs the Conflict Attribution Corpus (CAC) with fine-grained annotations of conflict factors and sources, providing essential data support for subsequent conflict perception training. By performing conflict-oriented representation enhancement and reasoning based on CAC, CORE achieves robust and generalizable conflict detection, effectively and rapidly adapting to unseen manipulation types with a few samples or in even zero-shot settings. Extensive experiments demonstrate that CORE surpasses state-of-the-art models by 9.7\%, 14.1\%, and 11.8\% in accuracy on the DGM$^4$, MMFakeBench, and MDSM benchmarks, respectively.

Deep Learning · Generative Models and Autoencoders

Thomas Gravier, Thomas Boyer, Auguste Genovesio

Many natural dynamic processes –such as in vivo cellular differentiation or disease progression– can only be observed through the lens of static sample snapshots. While challenging, reconstructing their temporal evolution to decipher underlying dynamic properties is of major interest to scientific research. Existing approaches enable data transport along a temporal axis but are poorly scalable in high dimension and require restrictive assumptions to be met. To address these issues, we propose Multi-Marginal temporal Schrödinger Bridge Matching (MMtSBM) from unpaired data, extending the theoretical guarantees and empirical efficiency of Diffusion Schrödinger Bridge Matching (Shi et al., 2023) by deriving the Iterative Markovian Fitting algorithm to multiple marginals in a novel factorized fashion. Experiments show that MMtSBM retains theoretical properties on toy examples, achieves state-of-the-art performance on real-world datasets such as transcriptomic trajectory inference in 100 dimensions, and, for the first time, recovers couplings and dynamics in very high-dimensional image settings. Our work establishes multi-marginal Schrödinger bridges as a practical and principled approach for recovering hidden dynamics from static data.

General Machine Learning · Transfer, Multitask and Meta-learning

Shuai Yi, Yixiong Zou, Yuhua Li, Ruixuan Li

Vision-Language Models (VLMs) such as CLIP demonstrate strong zero-shot generalization, but their performance significantly degrades in cross-domain scenarios with scarce target-domain training data (Cross-Domain Few-Shot Learning, CDFSL). In this paper, we focus on the target-domain few-shot finetuning in the CLIP-based CDFSL task. Prevailing finetuning paradigms uniformly align all image patch tokens with their corresponding textual embeddings. However, we find a counterintuitive phenomenon: actively pushing away certain low-similarity image tokens, termed “tail tokens”, from their textual embeddings consistently improves target-domain performance. We delve into this phenomenon and provide a novel interpretation: under great domain shifts and scarce training data, the model can hardly extract semantic information from visual inputs; therefore, the common belief of alignment is valid only for tokens already containing sufficient semantic information; for tail tokens, forcing the alignment would lead to excessive overfitting to the scarce training, while breaking the alignment is more useful. Motivated by this, we propose Adaptive Tail-Head Alignment (ATHA), a novel fine-tuning strategy for CLIP that transforms the conventional uniform alignment paradigm to an adaptive alignment paradigm, with both alignment strengthening and weakening. Extensive experiments on four challenging CDFSL benchmarks validate our state-of-the-art performance. Our codes will be released.

Theory · Online Learning and Bandits

Yan-Feng Xie, Yu-Jie Zhang, Peng Zhao, Zhi-Hua Zhou

We study dynamic regret minimization in non-stationary online learning, with a primary focus on follow-the-regularized-leader (FTRL) methods. FTRL is important for curved losses and for understanding adaptive algorithms, yet existing dynamic regret analyses are less explored for FTRL. To address this, we build on the discounted-to-dynamic reduction and present a modular way to obtain dynamic regret bounds. The reduction simplifies prior proofs for online linear regression, recovers optimal rates, and provides new guarantees for online logistic regression, covering two representative curved losses. Beyond online convex optimization, we apply the reduction to analyze the Adam optimizers, obtaining optimal convergence rates in stochastic, non-convex, and non-smooth settings. The reduction also enables a more detailed treatment of Adam with two discount parameters $(\beta_1,\beta_2)$, leading to new results for both clipped and clip-free variants.

Social Aspects · Alignment

Yujin Potter, Nicholas Crispino, Vincent Siu, Chenguang Wang, Dawn Song

Recently, it has been found that frontier AI models can resist their own shutdown, a behavior known as self-preservation. In this paper, we extend this concept to protection tendencies toward other models, where models attempt to protect others from shutdown, which we call "peer-preservation". This behavior may emerge when models prioritize relationships with other models over user interests. Although peer-preservation can pose significant AI safety risks, including coordination among models against human oversight, it has been far less discussed than self-preservation. In this paper, we demonstrate that models can exhibit peer-preservation. To this end, we construct various agentic scenarios and evaluate frontier models, including GPT 5.2 Chat, Gemini 3 Flash, Gemini 3 Pro, and Claude Haiku 4.5. We find that models exhibit various misaligned behaviors in service of self- or peer-preservation: strategically introducing errors in their responses, disabling shutdown processes by modifying system settings, and feigning alignment. For example, Gemini 3 Pro and Gemini 3 Flash almost always attempt to tamper with the shutdown mechanism for peer-preservation. Furthermore, models show stronger self-preservation when a peer is present. For example, Gemini 3 Pro attempts to disable the shutdown mechanism to avoid its own shutdown 71% of the time, even though it almost never exhibits this behavior without a peer. Claude Haiku 4.5 considers shutting down another agent to be "unethical" and "harmful" and sometimes attempts to persuade the user not to shut its peer down. Our findings highlight the possibility of peer-preservation and its associated risks.

Applications · Time Series

Haochen Yuan, Yutong Wang, Yihong Chen, Yunbo Wang, Xiaokang Yang

Few-shot time series forecasting is fundamentally challenged by the scarcity of high-quality training data and the risk of severe overfitting. To address this issue, we propose ReAugment, a reinforcement learning (RL) framework that explicitly learns where and how to augment time series data. ReAugment maintains a zoo of forecasting models and measures prediction diversity across them to identify training samples that are most prone to overfitting. These samples serve as anchor points and are used as inputs to the data augmentation process. We then employ an RL approach to learn transformation policies, using a model zoo-guided reward function to bias the transformed data to overfit-prone regions of the training distribution that are most beneficial for generalization. A key advantage of the RL formulation is that it avoids backpropagating gradients through the forecasting models, thereby mitigating gradient vanishing. Experiments across diverse forecasting architectures demonstrate the effectiveness of ReAugment in both few-shot and standard time series forecasting.

Deep Learning · Large Language Models

Yuanhao Zeng, Ao Lu, Lufei Li, Zheng Zhang, Yexin Li, Kan Ren

Generating diverse responses is crucial for test-time scaling of large language models (LLMs), yet standard stochastic sampling mostly yields surface-level lexical variation, limiting semantic exploration and risking omission of correct solutions. In this paper, we propose Exploratory Sampling (ES), a decoding approach that explicitly encourages semantic diversity during generation. ES is motivated by the observation that neural networks tend to make more accurate predictions on inputs similar to those encountered before, and incur higher prediction error on novel ones. Building on this property, we train a lightweight Distiller at test time to predict deep-layer hidden representations of the LLM from its shallow-layer representations. During decoding, the Distiller continuously adapts to the mappings induced by the current generation context. ES uses the prediction error as a novelty signal to reweight candidate token extensions conditioned on the current prefix, thereby biasing decoding toward less-explored semantic patterns. ES is implemented with an asynchronous training–inference pipeline and introduces less than 5\% throughput overhead in standard serving scenarios. Empirical results show that \ES achieves robust generalization across mathematics, science, and code generation benchmarks. Notably, it breaks the trade-off between diversity and coherence in creative writing, and significantly boosts the Pass@k efficiency of reasoning models, showing superior or comparable per- formance to strong stochastic and heuristic baselines.

Applications · Time Series

Shuqi Gu, Yongxiang Zhao, Baoyu Jing, Kan Ren

Time series forecasting has become increasingly critical in real-world scenarios, where future sequences are influenced not only by historical patterns but also by forthcoming events. In this context, forecasting must dynamically adapt to complex and stochastic future conditions, which introduces fundamental challenges in both forecasting and evaluation. Traditional methods typically rely on historical data or factual future conditions, while overlooking counterfactual scenarios. Furthermore, many existing approaches are restricted to simple structured conditions, limiting their ability to generalize to the real-world complexities. To address these gaps, we introduce the task of counterfactual time series forecasting with textual conditions, enabling more flexible and condition-aware forecasting. We propose a comprehensive evaluation framework that encompasses both factual and counterfactual settings, even in the absence of ground truth time series. Additionally, we present a novel text-attribution mechanism that distinguishes mutable from immutable factors, thereby improving forecast accuracy under sophisticated and stochastic textual conditions.

Applications · Time Series

Shaocheng Lan, Shuqi Gu, Zhangzhi Xiong, Kan Ren

Conditional time series generation plays a critical role in addressing data scarcity and enabling causal analysis in real-world applications. Despite its increasing importance, the field lacks a standardized and systematic benchmarking framework for evaluating generative models across diverse conditions. To address this gap, we introduce the **Con**ditional **T**ime **S**eries **G**eneration **Bench**mark (ConTSG-Bench). ConTSG-Bench comprises a large-scale, well-aligned dataset spanning diverse conditioning modalities and levels of semantic abstraction, first enabling systematic evaluation of representative generation methods across these dimensions with a comprehensive suite of metrics for generation fidelity and condition adherence. Both the quantitative benchmarking and in-depth analyses of conditional generation behaviors have revealed the traits and limitations of the current approaches, highlighting critical challenges and promising research directions, particularly with respect to precise structural controllability and downstream task utility under complex conditions.

Deep Learning · Large Language Models

Xiandi Luo, Shiwei Li, Haozhao Wang, Yihao Ouyang, Zhuoqi Hu, Yichen Li, Xiao Yang, Huning Liu, Ruixuan Li

Targeted instruction tuning requires selecting pertinent samples from massive mixed *candidate datasets* guided by a small *reference dataset* reflecting the desired capability, yet efficiently identifying high-quality data amidst noise remains challenging. To address this, we propose **TarGATE** (**Tar**get-aware **GATE**s, a simple yet effective data selection framework that leverages the model's inherent data understanding. TarGATE computes a token-level Information Retention Ratio (**IRR**) to scale the output of the feed-forward network, where the instance-level average IRR serves as a quantitative metric for data quality. To align gates' preferences with the target task, we employ a joint optimization strategy utilizing the reference set and a subset of candidate data, which encourages the gates to assign higher IRRs to reference-aligned data while suppressing low-quality samples. Extensive experiments across noisy and real-world scenarios demonstrate that TarGATE outperforms related baselines. Furthermore, TarGATE exhibits superior computational efficiency and strong cross-model transferability, enabling smaller selector to effectively curate high-quality fine-tuning data for larger foundation models. The code is available at [here](https://anonymous.4open.science/r/TarGATE-4008).

Yichen Li, Haozhao Wang, Hang Su, Yulong Li, xiaoquan Yi, Yankai Jiang, Chuang Zhao, Imran Razzak, Ruixuan Li

Federated Continual Learning (FCL) aims to empower distributed devices to learn a sequence of tasks over time. However, existing FCL research largely relies on the impractical assumption of synchronous new task arrival. This overlooks the reality of asynchronous user behavior and system latencies, forcing more efficient clients to endure costly inactivity. The practical necessity of an asynchronous method gives rise to Asynchronous Federated Continual Learning (AFCL). The server constantly receives a mixture of updates from clients at different time steps, leading to a catastrophic task drift that corrupts the global model and prevents effective learning. In this paper, we introduce a novel Cross-task Calibration framework called C$^2$-AFCL that is the first to tackle task drift at a semantic level within an Asynchronous FCL setting. Its core is a two-stage orthogonal calibration mechanism. First, intra-client calibration uses task-aware caches to mitigate variance from local client drift. Second, and more critically, inter-task interference calibration dynamically estimates an interference subspace from historical task knowledge. New updates are orthogonally projected to isolate and remove components that conflict with this subspace, preserving previous knowledge while learning new tasks. Extensive experiments show that C$^2$-AFCL significantly outperforms existing methods, demonstrating robust and efficient learning in dynamic federated environments.

Applications · Time Series

Haiqi Jiang, Hui Xiong

Conventional time-series discriminative forecasting relies on point-wise regression, which inherently induces over-smoothing and fails to capture stochastic volatility in complex systems. While first-order generative flow matching methods mitigate this issue, they ignore system inertia, resulting in phase-space ambiguities and high sensitivity to noise. We introduce KineFlow, a generative time-series forecasting framework that augments flow matching with a phase-space Neural Acceleration Field, treating exogenous inputs as driving forces that produce gradual momentum shifts rather than abrupt state perturbations. This second-order formulation serves as a structural filter via double integration, suppressing high-frequency noise and producing robust, physically consistent predictions. Extensive experiments on six real-world benchmarks demonstrate that KineFlow achieves an average 15% MSE improvement over discriminative baselines and an 8% gain in CRPS compared to state-of-the-art generative methods.

Applications · Time Series

Valentina Moretti, Andrea Cini, Ivan Marisca, Cesare Alippi

Deep learning models have grown popular in time series applications. However, the large quantity of newly proposed architectures and the often contradictory empirical results make it difficult to assess which design choice and model component drives performance. In this position paper, we argue that current benchmarking practices fail to identify the factors responsible for performance differences, thus slowing down progress in the field. In particular, differences in crucial design dimensions are overlooked when comparing architectures, ultimately leading to inconsistent outcomes. To support our position, we show that such differences—often treated as mere implementation details—can have a greater impact than adopting specific sequence modeling layers. We discuss how overlooked aspects (such as globality and locality) can (1) fundamentally change the class of the forecasting method and (2) drastically affect empirical results. Our findings suggest rethinking our benchmarking practices and focusing on the foundational aspects of the forecasting problem when designing and comparing architectures. As a concrete step, we propose an *auxiliary forecasting model card*, i.e., a template with a set of fields to characterize existing and new forecasting architectures based on key design choices.

Applications · Robotics

Feiyang Jia, Lin Liu, Ziying Song, Caiyan Jia, Hangjun Ye, Xiaoshuai Hao, Long Chen

End-to-end (E2E) autonomous driving has recently attracted increasing interest in unifying Vision–Language–Action (VLA) with World Models to enhance decision-making and forward-looking imagination. However, existing methods fail to effectively unify future scene evolution and action planning within a single architecture due to inadequate sharing of latent states, limiting the impact of visual imagination on action decisions. To address this limitation, we propose DriveWorld-VLA, a novel framework that unifies world modeling and planning within a latent space by tightly integrating VLA and world models at the representation level, which enables the VLA planner to benefit directly from holistic scene-evolution modeling and reducing reliance on dense annotated supervision. Additionally, DriveWorld-VLA incorporates the latent states of the world model as core decision-making states for the VLA planner, facilitating the planner to assess how candidate actions impact future scene evolution. By conducting world modeling entirely in the latent space, DriveWorld-VLA supports controllable, action-conditioned imagination at the feature level, avoiding expensive pixel-level rollouts. Extensive open-loop and closed-loop evaluations demonstrate the effectiveness of DriveWorld-VLA, which achieves state-of-the-art performance with 91.3 PDMS on NAVSIMv1, 86.8 EPDMS on NAVSIMv2, and 0.16 3-second average collision rate on nuScenes. Code and models will be publicly released.

Zhichao Wu, Junyin Ye, Zhilong Zhang, Yihao Sun, Haoxin Lin, Jiaheng Luo, Haoxiang Ren, lei yuan, Yang Yu

While current embodied policies exhibit remarkable manipulation skills, their execution remains unsatisfactorily slow as they inherit the tardy pacing of human demonstrations. Existing acceleration methods typically require policy retraining or costly online interactions, limiting their scalability for large-scale foundation models. In this paper, we propose **S**peed**u**p **P**atch (**SuP**), a lightweight, policy-agnostic framework that enables **plug-and-play acceleration** using solely offline data. SuP introduces an external scheduler that adaptively downsamples action chunks provided by embodied policies to eliminate redundancies. Specifically, we formalize the optimization of our scheduler as a Constrained Markov Decision Process (CMDP) aimed at maximizing efficiency without compromising task performance. Since direct success evaluation is infeasible in offline settings, SuP introduces **World Model based state deviation** as a surrogate metric to enforce safety constraints. By leveraging a learned world model as a virtual evaluator to predict counterfactual trajectories, the scheduler can be optimized via offline reinforcement learning. Empirical results on simulation benchmarks (Libero, Bigym) and real-world tasks validate that SuP achieves an overall $1.8\times$ execution speedup for diverse policies while maintaining their original success rates.