论文检索

General Machine Learning · Sequential, Network, and Time Series Modeling

Yuhang Pei, Fanchun Meng, Wenrui Wu, Tao Ren, Yifan Wang, Wei Ju, Chao Zheng, Xiao Luo

This paper studies the problem of single domain generalization in time series classification, which aims to learn a generalized time series classification model using a single source domain. This problem is highly challenging due to unreliable supervision from domain scarcity. Although current approaches employ generative models for data augmentation, these synthesized samples often suffer from low diversity and intrinsic noise, leading to weak generalization ability. Towards this end, we propose a novel approach named Context-driven Diffusion with Progressive Expansion (CURE) for single domain generalization in time series classification. The core of our CURE is to generate both semantic-aware and semantic-free contexts to strategically guide a conditional diffusion model for informative data expansion. In particular, our CURE first conducts representation disentanglement to extract semantic-aware and semantic-free representations from source data. To enhance generalizability through data synthesis, we not only retrieve reference time series trajectories with similar semantics for semantic-aware contexts, but also utilize adversarial strategies to learn semantic-free contexts. These contexts are integrated as joint conditions for a diffusion model, enabling diverse and reliable virtual data. To enhance expansion adaptability and stable optimization, we progressively update our semantic-free contexts via a memory bank and measure boundary properties for dynamic data filtering. Comprehensive experiments on benchmark datasets validate the effectiveness of the proposed CURE in comparison to extensive baselines. Our code is available at https://anonymous.4open.science/r/cure_9C6E/.

Thai Khanh Nguyen, Ngoc Bich Uyen Vo, Thieu Vo, Tan Nguyen, Cuong Pham

State-space models (SSMs) have emerged as efficient linear-time alternatives to attention for long-sequence modeling. However, existing SSMs often suffer from instability and memory degradation over extended horizons due to poorly conditioned first-order updates and uncontrolled spectral geometry. We introduce MuonSSM, a general framework that stabilizes SSM training by explicitly conditioning the geometry of memory updates rather than the recurrent transition matrix. MuonSSM augments standard SSMs with a momentum-based pathway and lightweight Newton–Schulz iterations on low-rank input injections, yielding approximately norm-preserving and spectrally balanced updates while preserving parallel scan complexity. Theoretical analysis demonstrates substantial improvements in gradient propagation and mitigation of vanishing gradients over long horizons. Extensive experiments across language, vision, and time-series benchmarks show consistent gains in accuracy, robustness, and long-context performance when integrated into diverse SSM backbones. These results establish geometric conditioning of updates as a principled pathway to stable, scalable sequence modeling.

Applications · Time Series

Hunjae Lee, Corey Clark

Standard attention mechanisms in transformers employ static token representations that remain unchanged across all pair-wise computations in each layer. This limits their representational alignment with the potentially diverse dynamics of each token-pair interaction. While they excel in domains with relatively homogeneous relationships, standard attention may be inadequate in capturing heterogeneous inter-channel dependencies of multivariate time series (MTS) data where different channel-pair interactions within a single system may be governed by entirely different physical laws or temporal dynamics. To better align the attention mechanism for such domain phenomena, we propose attention with dynamic relational priming (prime attention). Prime attention modulates token representations for each token-pair, optimizing each pair-wise interaction for that specific relationship. Our results demonstrate that prime attention consistently outperforms standard attention across benchmarks, achieving up to 6.5\% improvement in forecasting accuracy. In addition, prime attention achieves comparable performance using up to 40\% less sequence length compared to standard attention, demonstrating its superior relational modeling capabilities and potential for data efficiency.

Deep Learning · Sequential Models, Time series

Lars Graf, Thomas Ortner, Stanisław Woźniak, Angeliki Pantazi

Existing time series foundation models (TSFMs), often based on transformer variants, lack adaptability to different sampling rates, struggle with generalization across varying context and target lengths and are computationally inefficient. We introduce FlowState, a novel TSFM architecture that achieves sampling-rate–equivariant forecasting through a unified design that pairs a state space model (SSM) encoder with a functional basis decoder (FBD). This design enables continuous-time modeling and dynamic time-scale adjustment, allowing FlowState to inherently generalize across all possible temporal resolutions, and dynamically adjust the forecasting horizons without retraining. We further propose an efficient pretraining strategy that improves robustness and accelerates training. Despite being one of the smallest TSFMs, FlowState achieves state-of-the-art results on the widely used GIFT-Eval benchmark, while demonstrating superior adaptability to unseen sampling rates. Our detailed analyses confirm the effectiveness of its components, and we demonstrate its unique ability to adapt to varying input sampling rates.

Shiyu Wang, Yuchen Fang, Juntong Ni, Ziyi Zhang, Baichuan Mo, Xinyue Zhong, Chengxin Wang, Zhou Ye, Yang Xiang

Real-world multivariate time series are inherently in flux: different variables evolve asynchronously and interact in complex, time-varying ways, yet accurate forecasting requires these dispersed signals to converge into a single unified prediction. This structural mismatch between dynamic, heterogeneous inputs and a unified forecasting objective poses a fundamental challenge for building general-purpose multivariate forecasting models, especially in zero-shot and large-scale settings. To this end, inspired by the idea that "all rivers run into the sea", we propose ConFlux, a general-purpose foundation model for multivariate time-series forecasting by learning to adaptively integrate cross-channel information under a unified forecasting objective. Specifically, ConFlux first reorders variables to reduce cross-variable entanglement, then aggregates adjacent variables into compact patches that can be processed by a Vision Transformer-style architecture. This design shortens the effective context, reduces attention complexity, and provides a unified token representation for pre-training and downstream tasks. Experiments on 25 public datasets show that ConFlux achieves state-of-the-art performance in zero-shot, fine-tuning, and from-scratch settings, while offering faster inference and lower memory usage.

Applications · Time Series

Yangyou Liu, Zezhi Shao, Xinyu Chen, Hu Chen, Fei Wang, Yuankai Wu

Time series forecasting under non-stationarity faces a fundamental tension between capturing stable representations and adapting to distribution shifts. Existing methods implicitly rely on static historical assumptions, leading to a critical failure mode we term Phase Amnesia, where models become blind to the evolving global context. To resolve this, we formalize non-stationary dynamics through three physical hypotheses: Wold decomposition, dynamical phase evolution, and heteroscedastic manifold generation. These principles inspire PULSE, a physics-informed, plug-and-play framework adopting a Disentangle--Evolve--Simulate design philosophy. Specifically, PULSE utilizes phase-anchored disentanglement to resolve optimization interference caused by dominant trends, employs a Phase Router to actively generate future trajectories, and introduces Statistic-Aware Mixup (SAM) to ensure robustness against out-of-distribution volatility. Empirically, PULSE enables a simple MLP backbone to consistently outperform state-of-the-art Transformers across 12 real-world benchmarks. This validates that a correct physics-informed inductive bias is far more critical than raw architectural complexity for non-stationary forecasting.

Applications · Time Series

Han Zhang, Nenggan Zheng

Long-horizon motion prediction under external commands is challenged by latent decision uncertainty, where the internal states governing future behavior are unobservable and evolve stochastically over time. This issue is particularly pronounced in biological agents, whose motion trajectories reflect decision-making processes rooted in underlying cognitive states. To address these challenges, we propose CogSDE, a formulation of the controlled stochastic differential equation (SDE) for modeling instruction-driven latent decision dynamics. The drift term in the SDE incorporates a dual-channel control modulation mechanism, enabling external commands to modulate the latent state evolution. The diffusion term in the SDE adopts a state-dependent operator to model intrinsic uncertainty in latent decision dynamics. Furthermore, we establish a theoretical upper bound on the long-horizon prediction of CogSDE through dissipativity-based analysis. Experiments demonstrate that CogSDE consistently improves predictive accuracy in long-horizon motion generation. Importantly, predicted trajectories remain well aligned with control instructions over extended horizons, a property widely recognized as challenging in long-horizon motion prediction.

Applications · Time Series

Guikang Du, Haoran Li, Xinyu Liu, Zhibo Zhang, Xiaoli Gong, Jin Zhang

Cross-subject generalization in biomedical time-series (BTS) refers to training on data from some subjects and testing on unseen subjects. The key challenge is to suppress subject-specific variability in BTS representations. Most existing methods implicitly suppress the variability through model building or subject adversarial learning, but rarely model it explicitly. We introduce \textit{\textbf{spectral drift}} as a new perspective to characterize subject-specific variability. Specifically, BTS signals under the same label often share consistent oscillatory structure, yet exhibit subject-dependent magnitude or phase shifts in specific frequency components, which we interpret as subject-specific variability. Building on this insight, we propose \textbf{BioFormer}. At its core is a Frequency-Band Alignment Module (FBAM) that generates band-wise modulation factors from the spectral distribution and adaptively adjusts amplitude and phase to align spectral structure, thereby mitigating variability. We further pair FBAM with Sample-Conditional Layer Normalization, which infers normalization parameters from intrinsic signal statistics rather than subject identity, stabilizing cross-subject representations. Extensive experiments on six datasets demonstrate that BioFormer outperforms 12 baselines, yielding absolute F1-score improvements of 6\%.

Deep Learning · Large Language Models

Zikai Xiao, Jianhong Tu, Chuhang Zou, Yuxin Zuo, Zhi Li, Peng Wang, Bowen Yu, Fei Huang, Junyang Lin, Zuozhu Liu

Web agents require massive trajectories to generalize, yet real-world training is constrained by network latency, rate limits, and safety risks. We introduce \textbf{WebWorld} series, the first open-web simulator trained at scale. While existing simulators are restricted to closed environments with thousands of trajectories, WebWorld leverages a scalable data pipeline to train on 1M+ open-web interactions, supporting reasoning, multi-format data, and long-horizon simulations of 30+ steps. For intrinsic evaluation, we introduce WebWorld-Bench with dual metrics spanning nine dimensions, where WebWorld achieves simulation performance comparable to Gemini-3-Pro. For extrinsic evaluation, Qwen3-14B trained on WebWorld-synthesized trajectories improves by +9.2\% on WebArena, reaching performance comparable to GPT-4o. WebWorld enables effective inference-time search, outperforming GPT-5 as a world model. Beyond web simulation, WebWorld exhibits cross-domain generalization to code, GUI, and game environments, providing a replicable recipe for world-model construction.

Applications · Time Series

Ruituo Wu, Hongyu Zhang, Qiang Wang, Jiawei Du, Wei Cui, Ce Zhu, Bing Li

Learning from continuous-time trajectories requires modeling multivariate sensor measurements generated by underlying physical or dynamical processes. Under extreme data compression and heterogeneous sampling, directly optimizing synthetic signals as discrete sample values becomes fundamentally misaligned with the underlining \emph{continuous-time physical processes}, often producing high-frequency, non-physical artifacts that overfit specific models and break reuse across architectures and sampling rates. We propose CHESS, a \emph{function-first} synthesis framework shifts optimization from discrete samples to underlying continuous-time signal trajectories. CHESS injects physics-induced structure by jointly enforcing low-rank spatial coherence and piecewise Chebyshev polynomial temporal parameterization, constraining synthesis to a physically meaningful function manifold. We provide theoretical analysis establishing explicit smoothness and stability guaranties. Experiments on diverse sensor testbeds under the dataset distillation protocol demonstrate CHESS consistently outperform state-of-the-art methods with a compression ratios up to $133\times$ for each synthetic sample. Furthermore, CHESS exhibits strong cross-architecture generalization and enables zero-shot adaptation across different sampling resolutions.

Social Aspects · Accountability, Transparency, and Interpretability

Christodoulos Kechris, Jonathan Dan, David Atienza

Traditional saliency map methods, popularized in computer vision, highlight individual points (pixels) of the input that contribute the most to the model's output. However, in time series, they offer limited insights, as semantically meaningful features are often found in other domains. We introduce Cross-domain Integrated Gradients, a generalization of Integrated Gradients. Our method enables feature attributions in any domain that can be formulated as an invertible, differentiable transformation of the time domain. Crucially, our derivation extends the original Integrated Gradients into the complex domain, enabling frequency-based attributions. We provide the necessary theoretical guarantees, namely, path independence and completeness. We validate our method via controlled experiments with mechanistic analysis, quantitative faithfulness tests, and real-world case studies. Our approach reveals interpretable, problem-specific attributions that time-domain methods cannot capture in three real-world tasks across a variety of model architectures, machine-learning tasks, and cross-domain transforms: frequency-based attribution for a regression task in wearable heart rate extraction, independent component analysis in a classification task for electroencephalography-based seizure detection, and seasonal-trend decomposition for a forecasting problem with a zero-shot time-series foundation model. We release an open-source TensorFlow/PyTorch library to enable plug-and-play cross-domain explainability for time-series models. These results demonstrate the ability of Cross-Domain Integrated Gradients to provide semantically meaningful insights into time-series models that are impossible to achieve with traditional saliency in the time domain.

General Machine Learning · Sequential, Network, and Time Series Modeling

Taiki Miyagawa, Akinori F. Ebihara

We propose non-parametric estimators for the average run length (ARL) and average detection delay (ADD) in quickest changepoint detection (QCD) under finite and irregular sequence lengths. Although ARL and ADD are widely used as optimality criteria in theoretical and simulation studies, their application to real-world datasets is hindered by limited and irregular sequence lengths. To address this issue, we propose non-parametric estimators for the ARL and ADD, termed _KM-ARL and KM-ADD_, by drawing an analogy between QCD and survival analysis to model detection probabilities under sequence truncation. We derive estimation bias bounds and prove that they are asymptotically unbiased unless extrapolation is required. Experiments on simulated and real-world datasets demonstrate their practical utility, enhancing robustness against limited and irregular sequence lengths, improving interpretability, and facilitating empirical, intuitive model selection. Our Python code is provided in the supplementary material, offering ready-to-use implementations for practitioners.

Applications · Computer Vision

Zining Wang, Zhihang Yuan, Yingjie Zhai, Wenshuo Li, Han Shu, Ruihao Gong, Jinyang Guo, Xianglong Liu

Emerging Omni-modal Large Language Models (OmniLLMs) enable real-time interaction across video, audio, and text but suffer from prohibitive computational costs due to the quadratic complexity of processing continuous streaming inputs. Existing token compression strategies remain suboptimal as they typically rely on biased modality-centric priors or enforce uniform retention policies, neglecting the heterogeneity across layers and the critical role of cross-modality alignment. To address these challenges, we propose OmniFit, a training-free framework that decouples interaction profiling from inference execution. OmniFit incorporates Layer-Adaptive Heterogeneity Profiling (LAHP) to dynamically allocate computational budgets based on layer-wise redundancy and modality preferences, preserving tokens according to the characteristics of each layer. Furthermore, we introduce Alignment-Rectified Token Selection (ARTS), a lightweight mechanism that efficiently identifies tokens semantically aligned with cross-modal cues. Extensive experiments on 3 model series across 10 benchmarks demonstrate that OmniFit establishes a new Pareto frontier, retaining 98\% of model performance with only 20\% token usage and achieves up to 2.31$\times$ end-to-end inference speedup and 2.5$\times$ VRAM saving, significantly outperforming state-of-the-art methods.

Applications · Time Series

Jiahui Zhou, Dan Li, Boxin Li, Xiao Zhang, Erli Meng, Lin Li, Zhuomin Chen, Jian Lou, See-Kiong Ng

Time series is a pervasive data type across various application domains, rendering the reasonable solving of diverse time series tasks a long-standing goal. Recent advances in large language models (LLMs), especially their reasoning abilities unlocked through reinforcement learning (RL), have opened new opportunities for tackling tasks with long Chain-of-Thought (CoT) reasoning. However, leveraging LLM reasoning for time series remains infancy, hindered by the absence of carefully curated time series CoT data for training, limited data efficiency caused by underexplored data scheduling, and the lack of RL algorithms tailored for exploiting such time series CoT data. In this paper, we introduce VeriTime, a framework that tailors LLMs for time series reasoning through data synthesis, data scheduling, and RL training. First, we propose a data synthesis pipeline that constructs a time series–text multimodal dataset with process-verifiable annotations. Second, we design a data scheduling mechanism that arranges training samples according to a principled hierarchy of difficulty and task taxonomy. Third, we develop a two-stage reinforcement finetuning featuring fine-grained, multi-objective rewards that leverages verifiable process-level CoT data. Extensive experiments show that VeriTime substantially boosts LLM performance across diverse time series reasoning tasks. Notably, it enables compact 3B–4B models to achieve reasoning capabilities on par with or exceeding those of larger proprietary LLMs.

Applications · Time Series

Maximilian Mauel, Johannes Hübers, David Berghaus, Patrick Seifner, Ramses J Sanchez

Ordinary differential equations (ODEs) are central to scientific modelling, but inferring their vector fields from noisy trajectories remains challenging. Current approaches such as symbolic regression, Gaussian process (GP) regression, and Neural ODEs often require complex training pipelines and substantial machine learning expertise, or they depend strongly on system-specific prior knowledge. We propose FIM-ODE, a pretrained Foundation Inference Model that *amortises* ODE inference by predicting the vector field directly from noisy trajectory data *in a single forward pass*. We pretrain FIM-ODE on a prior distribution over ODEs with low-degree polynomial vector fields and represent the target field with neural operators. FIM-ODE achieves strong *zero-shot* performance, matching and often improving upon ODEFormer, a recent pretrained symbolic baseline, across a range of regimes despite using a simpler training prior. Pretraining also provides a strong initialisation for finetuning, enabling fast and stable adaptation that outperforms modern neural and GP baselines without requiring machine learning expertise. We release pretrained weights with the supplementary material.

Probabilistic Methods · Bayesian Models and Methods

John-Joseph Brady, Nikolas Nüsken, Yunpeng Li

Latent state space systems are ubiquitous in statistical modelling, arising naturally when a time-series is observed through a noisy measurement function, however training deep state space models (DSSM) at scale remains difficult. Two largely distinct strategies and literatures have developed around the training of DSSMs. Firstly, auto-encoding DSSMs train generative DSSMs by optimising a variational lower bound. Secondly, DSSMs trained by back-propagating the outputs of a classical sequential Monte Carlo algorithm (SMC). Such approaches can train DSSMs for discriminative as well as generative tasks, however, due to the sequentiality of their forward pass, scale poorly on modern hardware. We propose a new training method _Parallel Variational Monte Carlo_ (PVMC) that bridges the gap between the paradigms, and can be used robustly to train DSSMs for both discriminative and generative tasks. Our method achieves state-of-the-art or better results on a set of baseline experiments and trains $10\times$ faster than the fastest competing SMC approach.

Deep Learning · Sequential Models, Time series

Leon Götz, Marcel Kollovieh, Stephan Günnemann, Leo Schwinn

Existing time series tokenization methods predominantly encode a constant number of samples into individual tokens. This inflexible approach can generate excessive tokens for even simple patterns like extended constant values, resulting in substantial computational overhead. Inspired by the success of byte pair encoding, we propose the first pattern-centric tokenization scheme for time series analysis. Based on a discrete vocabulary of frequent motifs, our method merges samples with underlying patterns into tokens, compressing time series adaptively. Exploiting our finite set of motifs and the continuous properties of time series, we further introduce conditional decoding as a lightweight yet powerful post-hoc optimization method, which requires no gradient computation and adds no computational overhead. On recent time series foundation models, our motif-based tokenization improves forecasting performance by 36% and boosts efficiency by 1990% on average. Conditional decoding further reduces MSE by up to 44%. In an extensive analysis, we demonstrate the adaptiveness of our tokenization to diverse temporal patterns, its generalization to unseen data, and its meaningful token representations capturing distinct time series properties, including statistical moments and trends.

Applications · Time Series

Jihye Na, Patara Trirat, Chanyoung Park, Jae-Gil Lee

Recent studies have attempted to fine-tune time-series foundation models to enhance a target dataset's forecasting performance. However, these approaches proceed without a clear criterion for identifying complex datasets that require fine-tuning due to performance degradation in zero-shot forecasting. To distinguish datasets that are more challenging from standard benchmarks, we introduce data-driven temporal complexity and multichannel complexity. *Temporal complexity* captures the difficulty of identifying distinct patterns by quantifying spectral entropy in the frequency domain, while *multichannel complexity* captures inter-channel dependencies by measuring the channel information flow impacting predictive uncertainty. These metrics serve as *effective proxies for performance gains* achievable through fine-tuning. Based on the two metrics, we develop *Time-PEFT*, a parameter-efficient fine-tuning framework that incorporates a frequency adapter for top-$k$ filtering and a channel adapter for multichannel modeling. *Time-PEFT* is shown to significantly improve forecasting performance by up to 2.51 times compared with existing fine-tuning techniques on complex datasets.

Deep Learning · Sequential Models, Time series

Egor Serov, Ilya Kuleshov, Alexey Zaytsev

Neural Controlled Differential Equations (Neural CDEs) provide a powerful continuous-time framework for sequence modeling, yet the roughness of the driving control path often restricts their efficiency. Standard splines introduce high-frequency variations that force adaptive solvers to take excessively small steps, driving up the Number of Function Evaluations (NFE). We propose a novel approach to Neural CDE path construction that replaces exact interpolation with Kernel and Gaussian Process (GP) smoothing, enabling explicit control over trajectory regularity. To recover details lost during smoothing, we propose an attention-based Multi-View CDE (MV-CDE) and its convolutional extension (MVC-CDE), which employ learnable queries to inform path reconstruction. This framework allows the model to distribute representational capacity across multiple trajectories, each capturing distinct temporal patterns. Empirical results demonstrate that our method, MVC-CDE with GP, achieves state-of-the-art accuracy while significantly reducing NFEs and total inference time compared to spline-based baselines.

Deep Learning · Foundation Models

Jingru Fei, Kun Yi, Alex Wang, Qingsong Wen, Xiangxiang Zhu, Wei Fan

Time series foundation models rely on large-scale pretraining over diverse datasets across domains, yet their heterogeneity in temporal patterns could hinder the effectiveness of training and learning transferable time series representations. Inspired a fundamental concept, normalized power spectral density (PSD) in signal processing, we assume harmonizing datasets via PSDs in the spectral domain could reduce mismatches and enhance pretraining. We then go beyond the direct intractable minimization optimization and innovatively reformulate it as a principled \textit{harmonization} approach. Specifically, we propose \textit{Harmonizer}, a module that reshapes spectral structures and implicitly harmonizing PSDs across datasets, which theoretically corresponds to a shared reparameterization of second-order temporal correlations. Our theoretical analysis further reveals token interactions with Harmonizer can be efficiently mediated by a compact set of resonators, motivating a \textit{HarmonicAttention} design that performs self-attention in a low-dimensional interaction space. Then, we propose \textit{Olivia}, a novel time series foundation model built upon these harmonization mechanisms. Extensive experiments on two large-scale benchmarks (TSLib and GIFT-Eval) and extra 6 datasets from GluonTS, demonstrate Olivia consistently achieves state-of-the-art performance under zero-shot, few-shot, and full-shot forecasting scenarios.