论文检索

Deep Learning · Sequential Models, Time series

Tian Lan, Hao Le, Jinbo Li, Wenjun He, Meng Wang, Chenghao Liu, Chen Zhang

TSAD is a critical task, but developing models that generalize to unseen data in a zero-shot manner remains a major challenge. Prevailing foundation models for TSAD predominantly rely on reconstruction-based objectives, which suffer from a fundamental objective mismatch and representation conflict: they tend to memorize static patterns from training data, struggling to identify subtle anomalies while often misinterpreting complex normal patterns in unseen domains. To overcome these limitations, we introduce TimeRCD, a novel foundation model for TSAD built upon a new pre-training paradigm: Relative Context Discrepancy (RCD). Instead of reconstructing inputs based on fixed priors, TimeRCD is explicitly trained to adaptively identify anomalies by contrasting the query with its surrounding context. This relational approach, implemented with a standard Transformer architecture, enables the model to infer normality on-the-fly and capture contextual shifts indicative of anomalies that reconstruction-based methods often miss. To empower this paradigm, we develop a large-scale, diverse synthetic corpus with context-dependent anomaly labels, providing the rich supervisory signal necessary for effective pre-training. Extensive experiments demonstrate that TimeRCD significantly outperforms existing general-purpose and anomaly-specific foundation models in zero-shot TSAD across diverse datasets. Our results validate the superiority of the RCD paradigm and establish a new, effective path toward building robust and generalizable foundation models for time series anomaly detection. The code is available in \url{https://anonymous.4open.science/r/TimeRCD-5BE1/}

General Machine Learning · Sequential, Network, and Time Series Modeling

Cheng Yu, Zhoufan Zhu, Ke Zhu

Conformal prediction is challenging for time series with the time-varying conditional distributions. Existing sequential conformal methods can yield volatile, non-nested prediction intervals due to noisy tail conditional quantile estimation and quantile crossing issue. To overcome this challenge, we construct the prediction intervals for time series via a novel method called Conditional Quantile Adjusted Conformal Prediction (CQACP), which stabilizes sequential conformal calibration by modeling the conditional quantile curve of nonconformity score. At each time step, CQACP evaluates a base conditional quantile learner on a grid of quantile levels, and fits a Cornish-Fisher approximation parameterized by conditional moments of nonconformity score with monotonicity constraints. Asymptotically, we prove the conditional validity of the prediction interval under serial dependence and show improved conditional quantile estimation accuracy. Experiments on multiple real-world datasets demonstrate that CQACP maintains accurate coverage and produces smooth, narrow, and nested prediction intervals across different significance levels and prediction models.

General Machine Learning · Sequential, Network, and Time Series Modeling

Yiheng Gu, Xiufan Yu

This paper proposes a power-enhanced panel Granger causality test (PE-PGCT) for assessing the Granger non-causality in heterogeneous and potentially high-dimensional panel data. Building on any existing panel Granger non-causality test, we show, both theoretically and empirically, that the proposed PE-PGCT boosts its power substantially. The power gains are particularly significant in situations of high-dimensional panels when the cross-sectional dimension exceeds the time dimension, as well as under sparse alternatives when the signals are sparsely distributed across panel units. We establish rigorous theoretical guarantees on the asymptotic behavior of the proposed power enhancement component, demonstrating attractive power enhancement properties that it induces negligible size distortion under the null hypothesis while delivering significant power gain under the alternatives. The empirical performances are illustrated via extensive simulation studies, as well as a real-world application.

General Machine Learning · Sequential, Network, and Time Series Modeling

Junyao Hong, Zesheng Lai, Xinyi Xiao, Suyang Zhou, Aodong Shen, Youyong Kong

Knowledge Distillation (KD) promises to bridge the gap between the high computational costs of Transformer-based models and the expressiveness limitations of linear models in long-term time series forecasting. Existing time series distillation methods inherit the computer vision paradigm, constraining student models by minimizing point-wise prediction matching (output-level distillation) errors. However, blindly mimicking teacher predictions, which are often uncertain, can induce negative transfer. To address this, we propose Dynamic Structural Distillation (DSD), a robust framework that goes beyond the prediction-matching paradigm. First, we design LMP-Net, leveraging manifold expansion to project features into a high-dimensional latent space, alleviating the expressiveness bottleneck while preserving lightweight inference. Second, to address architectural mismatch, we propose Dual Manifold Alignment, employing Similarity-Preserving Knowledge Distillation (SPKD) and Optimal Transport (OT) to align features at the topological and geometric levels, respectively. Finally, we introduce Regime-Aware Adaptive Distillation (RAAD) to mitigate teacher misguidance via a dataset-level regime prior and a confidence-based adaptive gating mechanism. Extensive experiments on five benchmarks validate that DSD is compatible with diverse Transformer-based teachers, mitigating negative transfer while achieving a favorable accuracy--efficiency trade-off. An anonymized implementation is available at https://anonymous.4open.science/r/DSD-master-4B8F.

General Machine Learning · Sequential, Network, and Time Series Modeling

Beinan Xu, Andy Song, Jiti Gao, Feng Liu

We introduce Equilibrium State Estimation (ESE), a novel paradigm for simultaneous prediction, where multiple interacting systems require separate yet coordinated forecasts. Such scenarios often arise in real-world such as economics and healthcare modeling. Unlike existing approaches that predict one system at a time, ESE forecasts all systems in a single pass. It first estimates the equilibrium state across systems, then generates holistic forecasts based on the difference between the current state and the estimated equilibrium. Extensive experiments on synthetic and real-world datasets, including currency exchange and COVID-19 spread modeling, demonstrate that ESE is at least as accurate as state-of-the-art (SOTA) methods while being significantly faster. In addition, ESE integrates seamlessly with conventional predictors, combining their accuracy with its exceptional efficiency and delivering a 10–70× speedup. With linear-time complexity, ESE scales far better than SOTA methods as the number of systems increases. Moreover, it remains accurate under diverse perturbations, establishing ESE as a fast, generalizable, robust, and scalable multi-prediction method. Source code and data are available at https://anonymous.4open.science/r/ESE-C339.

General Machine Learning · Sequential, Network, and Time Series Modeling

Nitin Bisht, Xiuwen Gong, Huan Huo, Guandong Xu

Modern recommender systems are typically trained offline and deployed with parameters held fixed between periodic refreshes, yet user behavior can evolve substantially during deployment. This can cause ranking utility to degrade over time and makes it difficult to provide formal guarantees about recommendation quality. We propose **CARE**, an adaptive calibration framework that wraps an arbitrary backbone recommender and outputs variable-size recommendation sets with finite-sample performance guarantees over interaction streams. CARE combines (i) a loss-based monitoring module that localizes behavioral changes and triggers threshold recalibration, and (ii) an online aggregation rule that promotes compact recommendation sets by dynamically reweighting candidate set predictors. We provide theoretical results establishing finite-sample guarantees for utility-based risk control and bounds on the expected set size relative to the best constituent predictor. Experiments across multiple datasets and backbone models demonstrate that CARE improves robustness and maintains compact recommendation sets while preserving the desired statistical guarantees. The code and implementation are available in https://anonymous.4open.science/r/CARE-FCBD.

Deep Learning · Sequential Models, Time series

Sangjun Park

Recent artificial neural networks has demonstrated remarkable capabilities across various tasks, raising expectations for Human-Level AI (HLAI). This position paper argues that integrating explicit memory is instrumental in advancing current AI towards HLAI. The key reason is that the underlying learning mechanism of artificial neural networks bears a notable resemblance to implicit memory of the basal ganglia. However, higher-order cognitive functions necessary for HLAI, such as long-term strategic planning, metacognition, and symbolic reasoning, heavily rely on the hippocampal explicit memory and cannot arise solely from implicit statistical learning. Based on this perspective, we define the computational requirements for artificial explicit memory systems, with the aim of fostering further research and laying the groundwork for explicit memory integration.

Deep Learning · Sequential Models, Time series

Yubin Kim, Viresh Pati, Jevon Twitty, Vinh Pham, Shihao Yang, Jiecheng Lu

Transformer architectures have established strong baselines in time series forecasting, yet they typically rely on positional encodings that assume uniform, index-based temporal progression. However, real-world systems, from shifting financial cycles to elastic biological rhythms, frequently exhibit ``time-warped'' dynamics where the effective flow of time decouples from the sampling index. In this work, we first formalize this misalignment and prove that rotary position embedding (RoPE) is mathematically incapable of representing non-affine temporal warping. To address this, we propose Symplectic Positional Embeddings (SyPE), a learnable encoding framework derived from Hamiltonian mechanics. SyPE strictly generalizes RoPE by extending the rotation group $\mathrm{SO}(2)$ to the symplectic group $\mathrm{Sp}(2,\mathbb{R})$, modulated by a novel input-dependent adaptive warp module. By allowing the attention mechanism to adaptively dilate or contract temporal coordinates end-to-end, our approach captures locally varying periodicities without requiring pre-defined warping functions. We implement this mechanism in StretchTime, a multivariate forecasting architecture that achieves state-of-the-art performance on standard benchmarks, demonstrating superior robustness on datasets exhibiting non-stationary temporal dynamics.

General Machine Learning · Sequential, Network, and Time Series Modeling

Yuhang Pei, Fanchun Meng, Wenrui Wu, Tao Ren, Yifan Wang, Wei Ju, Chao Zheng, Xiao Luo

This paper studies the problem of single domain generalization in time series classification, which aims to learn a generalized time series classification model using a single source domain. This problem is highly challenging due to unreliable supervision from domain scarcity. Although current approaches employ generative models for data augmentation, these synthesized samples often suffer from low diversity and intrinsic noise, leading to weak generalization ability. Towards this end, we propose a novel approach named Context-driven Diffusion with Progressive Expansion (CURE) for single domain generalization in time series classification. The core of our CURE is to generate both semantic-aware and semantic-free contexts to strategically guide a conditional diffusion model for informative data expansion. In particular, our CURE first conducts representation disentanglement to extract semantic-aware and semantic-free representations from source data. To enhance generalizability through data synthesis, we not only retrieve reference time series trajectories with similar semantics for semantic-aware contexts, but also utilize adversarial strategies to learn semantic-free contexts. These contexts are integrated as joint conditions for a diffusion model, enabling diverse and reliable virtual data. To enhance expansion adaptability and stable optimization, we progressively update our semantic-free contexts via a memory bank and measure boundary properties for dynamic data filtering. Comprehensive experiments on benchmark datasets validate the effectiveness of the proposed CURE in comparison to extensive baselines. Our code is available at https://anonymous.4open.science/r/cure_9C6E/.

Deep Learning · Sequential Models, Time series

Lars Graf, Thomas Ortner, Stanisław Woźniak, Angeliki Pantazi

Existing time series foundation models (TSFMs), often based on transformer variants, lack adaptability to different sampling rates, struggle with generalization across varying context and target lengths and are computationally inefficient. We introduce FlowState, a novel TSFM architecture that achieves sampling-rate–equivariant forecasting through a unified design that pairs a state space model (SSM) encoder with a functional basis decoder (FBD). This design enables continuous-time modeling and dynamic time-scale adjustment, allowing FlowState to inherently generalize across all possible temporal resolutions, and dynamically adjust the forecasting horizons without retraining. We further propose an efficient pretraining strategy that improves robustness and accelerates training. Despite being one of the smallest TSFMs, FlowState achieves state-of-the-art results on the widely used GIFT-Eval benchmark, while demonstrating superior adaptability to unseen sampling rates. Our detailed analyses confirm the effectiveness of its components, and we demonstrate its unique ability to adapt to varying input sampling rates.

General Machine Learning · Sequential, Network, and Time Series Modeling

Taiki Miyagawa, Akinori F. Ebihara

We propose non-parametric estimators for the average run length (ARL) and average detection delay (ADD) in quickest changepoint detection (QCD) under finite and irregular sequence lengths. Although ARL and ADD are widely used as optimality criteria in theoretical and simulation studies, their application to real-world datasets is hindered by limited and irregular sequence lengths. To address this issue, we propose non-parametric estimators for the ARL and ADD, termed _KM-ARL and KM-ADD_, by drawing an analogy between QCD and survival analysis to model detection probabilities under sequence truncation. We derive estimation bias bounds and prove that they are asymptotically unbiased unless extrapolation is required. Experiments on simulated and real-world datasets demonstrate their practical utility, enhancing robustness against limited and irregular sequence lengths, improving interpretability, and facilitating empirical, intuitive model selection. Our Python code is provided in the supplementary material, offering ready-to-use implementations for practitioners.

Probabilistic Methods · Bayesian Models and Methods

John-Joseph Brady, Nikolas Nüsken, Yunpeng Li

Latent state space systems are ubiquitous in statistical modelling, arising naturally when a time-series is observed through a noisy measurement function, however training deep state space models (DSSM) at scale remains difficult. Two largely distinct strategies and literatures have developed around the training of DSSMs. Firstly, auto-encoding DSSMs train generative DSSMs by optimising a variational lower bound. Secondly, DSSMs trained by back-propagating the outputs of a classical sequential Monte Carlo algorithm (SMC). Such approaches can train DSSMs for discriminative as well as generative tasks, however, due to the sequentiality of their forward pass, scale poorly on modern hardware. We propose a new training method _Parallel Variational Monte Carlo_ (PVMC) that bridges the gap between the paradigms, and can be used robustly to train DSSMs for both discriminative and generative tasks. Our method achieves state-of-the-art or better results on a set of baseline experiments and trains $10\times$ faster than the fastest competing SMC approach.

Deep Learning · Sequential Models, Time series

Leon Götz, Marcel Kollovieh, Stephan Günnemann, Leo Schwinn

Existing time series tokenization methods predominantly encode a constant number of samples into individual tokens. This inflexible approach can generate excessive tokens for even simple patterns like extended constant values, resulting in substantial computational overhead. Inspired by the success of byte pair encoding, we propose the first pattern-centric tokenization scheme for time series analysis. Based on a discrete vocabulary of frequent motifs, our method merges samples with underlying patterns into tokens, compressing time series adaptively. Exploiting our finite set of motifs and the continuous properties of time series, we further introduce conditional decoding as a lightweight yet powerful post-hoc optimization method, which requires no gradient computation and adds no computational overhead. On recent time series foundation models, our motif-based tokenization improves forecasting performance by 36% and boosts efficiency by 1990% on average. Conditional decoding further reduces MSE by up to 44%. In an extensive analysis, we demonstrate the adaptiveness of our tokenization to diverse temporal patterns, its generalization to unseen data, and its meaningful token representations capturing distinct time series properties, including statistical moments and trends.

Deep Learning · Sequential Models, Time series

Egor Serov, Ilya Kuleshov, Alexey Zaytsev

Neural Controlled Differential Equations (Neural CDEs) provide a powerful continuous-time framework for sequence modeling, yet the roughness of the driving control path often restricts their efficiency. Standard splines introduce high-frequency variations that force adaptive solvers to take excessively small steps, driving up the Number of Function Evaluations (NFE). We propose a novel approach to Neural CDE path construction that replaces exact interpolation with Kernel and Gaussian Process (GP) smoothing, enabling explicit control over trajectory regularity. To recover details lost during smoothing, we propose an attention-based Multi-View CDE (MV-CDE) and its convolutional extension (MVC-CDE), which employ learnable queries to inform path reconstruction. This framework allows the model to distribute representational capacity across multiple trajectories, each capturing distinct temporal patterns. Empirical results demonstrate that our method, MVC-CDE with GP, achieves state-of-the-art accuracy while significantly reducing NFEs and total inference time compared to spline-based baselines.

Deep Learning · Sequential Models, Time series

Bohan Wang, Zewen Liu, Lu Lin, Hui Liu, Li Xiong, Ming Jin, Wei Jin

Interpretable time series deep learning systems are often assessed by checking temporal consistency on explanations, implicitly treating this as evidence of robustness. We show that this assumption can fail: Predictions and explanations can be adversarially decoupled, enabling targeted misclassification while the explanation remains plausible and consistent with a chosen reference rationale. We propose TSEF (Time Series Explanation Fooler), a dual-target attack that jointly manipulates the classifier and explainer outputs. In contrast to single-objective misclassification attacks that disrupt explanation structure and spread attribution mass broadly, TSEF achieves targeted prediction changes while keeping explanations consistent with the reference. Across multiple datasets and explainer backbones, our results consistently reveal that explanation stability is a misleading proxy for decision robustness and motivate coupling-aware robustness evaluations for trustworthy time series deployments.

Deep Learning · Sequential Models, Time series

Hao Miao, Ni Zhang, Zefei Ning, Li Wang

Battery lifetime early prediction is crucial for safety assessment and decision planning. However, early-stage degradation signals are extremely weak and difficult to distinguish from noise. Existing methods primarily rely on denoising or signal decomposition, which risks losing critical degradation cues. In nature, most organisms exhibit binaural effect, exploiting differences between left and right auditory inputs to enhance perceptual reliability. Inspired by this, we propose DITING, a weak degradation listener for battery lifetime early prediction. DITING first employs optimal-transport-based selective matching to extract a robust health template from initial cycles for degradation representation. To manifest degradation signals from noise, we further design a tri-coupled degradation manifestation mechanism. By exploiting the randomness of noise, matched responses under symmetric coupling suppress stochastic fluctuations. Conversely, cumulative deviations driven by degradation propagate through the coupling to form stable bilateral discrepancies. This design effectively amplifies weak cues in the early stage for lifetime prediction. Experiments on multiple datasets demonstrate that DITING achieves state-of-the-art performance and provides more reliable early support for full-lifecycle battery management. The code is available at https://anonymous.4open.science/r/Batt_DITING.

Deep Learning · Sequential Models, Time series

Dai Shi, Lequan Lin, Andi Han, Luke Thompson, Jose Miguel Hernandez-Lobato, Zhiyong Wang, Junbin Gao

Stochastic differential equations (SDEs) and stochastic partial differential equations (SPDEs) are fundamental for modeling stochastic dynamics across the natural sciences and modern machine learning. Learning their solution operators with deep learning models promises fast solvers and new perspectives on classical learning tasks. In this work, we build on Wiener–chaos expansions (WCE) to design neural operator (NO) architectures for SDEs and SPDEs: we project driving noise paths onto orthonormal Wick–Hermite features and use NOs to parameterize the resulting chaos coefficients, enabling reconstruction of full trajectories from noise in a single forward pass. We also make the underlying WCE structure explicit for multi-dimensional SDEs and semilinear SPDEs by showing the coupled deterministic ODE/PDE systems governing these coefficients. Empirically, we achieve competitive accuracy across several tasks, including standard SPDE benchmarks and SDE-based diffusion one-step image sampling, topological graph interpolation, financial extrapolation, parameter estimation, and manifold SDE flood forecasting. These results suggest WCE-based neural operators are a practical and scalable approach to learning SDE/SPDE solution operators across domains.

Deep Learning · Sequential Models, Time series

Luke Thompson, Dai Shi, Lequan Lin, Junbin Gao, Andi Han

Neural rough differential equations (NRDEs) learn continuous-time dynamics from irregularly sampled sequences by encoding the input path with signature features, providing robustness to discretisation and sampling irregularity. However, existing NRDEs implicitly rely on algebraic identities that can fail in two important settings: *stochastic dynamics* interpreted in the Itô sense, and *dynamics evolving on manifolds* where curvature renders the effect of repeated derivatives order-dependent. In this work, we propose Branched Neural Rough Differential Equations (B-NRDEs), a unified framework that replaces geometric signature features with tree-based (branched) rough-path lifts, yielding models that remain well-defined under Itô noise and on manifolds. Building on these branched lifts, an Itô-consistent training objective is introduced via the branched signature kernel. We provide an efficient, autodifferentiable package *Stochastax* for computing branched (log-)signatures and solving (manifold) RDEs. Across various applications, including rough Bergomi volatility modelling, sim-to-real $\mathrm{SO}(3)$ dynamics forecasting, and SPD covariance dynamics, B-NRDE shows consistently strong results.

Deep Learning · Sequential Models, Time series

Md Mahmuddun Nabi Murad, Yasin Yilmaz

Early and accurate detection of anomalies in time-series data is critical due to the substantial risks associated with false or missed detections. While MLP-based mixer models have shown promise in time-series analysis, they do not maintain temporal causality during data processing. Moreover, real-world multivariate time series often contain numerous channels with diverse inter-channel correlations. Spurious correlations in the reconstructed time series lead to noisy representations, resulting in inaccurate anomaly detection. In addition, anomaly scoring methods that ignore temporal continuity can mislead sequential detection. To address these challenges, we propose a cluster-aware causal mixer for multivariate time-series anomaly detection. Channels are grouped into clusters based on their correlations, and each cluster is embedded through a dedicated embedding layer. A causal mixer is introduced to integrate information while maintaining temporal causality. We further develop a sequential anomaly-scoring method that accumulates evidence over time and refines anomaly boundaries. Our proposed model operates in an online fashion, making it suitable for real-time time-series anomaly detection. Experimental evaluations across six public benchmark datasets demonstrate that the proposed approach consistently achieves superior performance.

General Machine Learning · Sequential, Network, and Time Series Modeling

Dimitra Maoutsa

How can we learn the laws underlying the dynamics of stochastic systems when their trajectories are sampled sparsely in time? Existing methods either require temporally resolved high-frequency observations, or rely on geometric arguments that apply only to conservative systems, limiting the range of dynamics they can recover. Here, we present a new framework that reconciles these two perspectives by reformulating inference as a stochastic control problem. Our method uses geometry-driven path augmentation, guided by structure in the system’s invariant density to reconstruct likely trajectories and infer the underlying dynamics without assuming specific parametric models. Applied to overdamped Langevin systems, our approach accurately recovers stochastic dynamics even from severely undersampled data, outperforming existing methods in synthetic benchmarks. This work demonstrates the effectiveness of incorporating geometric inductive biases into stochastic system identification methods, with broad applications across physics, biology, and control.