论文检索

Deep Learning · Sequential Models, Time series

Alexis-Raja Brachet, Pierre-Yves Richard, Céline Hudelot

While deep learning is facing an homogenization across modalities led by Transformers, they are still challenged by shallow linear models in the time-series forecasting task. Our hypothesis is that models should learn a direct link from past to future data points, which we identify as a learning dynamics capability. We develop an original $\texttt{PRO-DYN}$ nomenclature to analyze existing models through the lens of dynamics. Two observations thus emerge: **1.** under-performing architectures learn dynamics at most partially, **2.** the location of the dynamics block at the model end is of prime importance. Our systemic and empirical studies both confirm our observations on a set of performance-varying models with diverse backbones. We propose a simple plug-and-play methodology guiding model designs and improvements.

Applications · Time Series

Chen-Hui Song, Shuoling Liu, Liyuan Chen

While deep learning has revolutionized financial forecasting through sophisticated architectures, the design of the supervision signal itself is rarely scrutinized. We challenge the canonical assumption that training labels must strictly mirror inference targets, uncovering the Label Horizon Paradox: the optimal supervision signal often deviates from the prediction goal, shifting across intermediate horizons governed by market dynamics. We theoretically ground this phenomenon in a dynamic signal-noise trade-off, demonstrating that generalization hinges on the competition between marginal signal realization and noise accumulation. To operationalize this insight, we propose a bi-level optimization framework that autonomously identifies the optimal proxy label within a single training run. Extensive experiments on large-scale financial datasets demonstrate consistent improvements over conventional baselines, thereby opening new avenues for label-centric research in financial forecasting.

Applications · Time Series

guanqun zhao, Yitong Liu, Jiaxuan Fang, Hongwen Yang, Yufei Mao

Standard Self-Supervised Learning (SSL) for Automatic Modulation Recognition (AMR) struggles with ineffective isotropic augmentations, spectral instability, and semantic drift. To address these challenges, we propose Dynamic-Consistency Contrastive Learning (DyCo-CL), a geometry-aware framework that couples Virtual Adversarial Augmentation (VAA) with a semantic consistency loss. We provide a theoretical analysis indicating that this strategy acts as an implicit spectral regularizer for the encoder, enabling stable manifold exploration. Complementing this, our Signal-Adaptive Swin Backbone with fixed-window attention improves structural stability by constraining attention locality, while a Hybrid Knowledge Fusion module anchors representations with physical priors. Experiments on RML benchmarks show that DyCo-CL achieves a 6.27% accuracy gain in 1-shot settings over prior methods.

Deep Learning · Sequential Models, Time series

Ruichao Guo, Xingyao Han, Wenshui Luo, Zhe Liu, Chen Gong, Hesheng Wang

Point forecasting for graph-structured multivariate time series is a fundamental problem, but rigorous uncertainty quantification for such predictions is still underexplored. Conformal prediction (CP) offers uncertainty estimation with a solid coverage guarantee under the exchangeability assumption, which requires the joint data distribution to be unchanged under permutation. However, in graph-structured time series, inherent cross-node coupling can violate the exchangeability condition, making direct application of CP unreliable. Inspired by the spectral graph theory, such coupling resides in global trends and can be characterized by the low-frequency components, while high-frequency components are nearly exchangeable. Therefore, we propose a novel concept named **S**pectral **G**raph **C**onditional **E**xchangeability (SGCE), which conditions exchangeable high-frequency components on low-frequency ones to preserve global trends and enable effective CP in the spectral domain. Based on SGCE, we further propose **S**pectral **C**onformal prediction via w**A**ve**LE**t transform (SCALE). SCALE uses graph wavelets to decompose low/high-frequency components and conformalizes high-frequency residuals via adaptive gating over a low-frequency embedding. Experimental results on real-world traffic datasets show that SCALE not only achieves valid coverage but also consistently improves the coverage-efficiency trade-off over the state-of-the-art CP methods.

Applications · Time Series

Xiaorui Wang, Fanda Fan, Chenxi Wang, Yuxuan Yang, Rui Tang, Kuoyu Gao, simiao pang, Yuanfeng Shang, Liu, Wanling Gao 等

Recent progress in time-series forecasting has led to rapidly increasing architectural complexity, yet many reported State-of-the-Art gains are statistically fragile or misattributed. We argue that progress requires a shift from model selection to modular attribution, identifying which components truly drive performance. We propose CombinationTS, a self-contained probabilistic evaluation framework that decomposes forecasting models into orthogonal modules—Input Transformation, Embedding, Encoder, and Decoder—and evaluates them under a shared evaluation condition space. By quantifying each component via marginalized effectiveness ($\mu$) and stability ($\sigma^2$), CombinationTS enables robust attribution beyond fragile point estimates. Through large-scale paired evaluation, we uncover the Identity Paradox: once the data view is well-designed, a parameter-free Identity encoder often matches or outperforms complex backbones. We further show that explicit structural priors introduced via input transformations yield a more favorable effectiveness–stability trade-off than increasing encoder complexity, establishing a principled baseline for architectural necessity. The code is available at https://anonymous.4open.science/r/CombinationTS.

Deep Learning · Algorithms

Hong-Kun Zhang, Xin Li, Sikun Yang, Zhihong Xia

A novel neural network inspired by Cauchy's integral formula, is proposed for function approximation tasks that include time series forecasting, missing data imputation, etc. Hence, the novel neural network is named CauchyNet. By embedding real-valued data into the complex plane, CauchyNet efficiently captures complex temporal dependencies, surpassing traditional real-valued models in both predictive performance and computational efficiency. Grounded in Cauchy's integral formula and supported by the universal approximation theorem, CauchyNet offers strong theoretical guarantees for function approximation. The architecture incorporates complex-valued activation functions, enabling robust learning from incomplete data while maintaining a compact parameter footprint and reducing computational overhead. Through extensive experiments in diverse domains, including transportation, energy consumption, and epidemiological data, CauchyNet consistently outperforms state-of-the-art models in predictive accuracy, often achieving a lower mean absolute error with fewer parameters. These findings highlight CauchyNet’s potential as an effective and efficient tool for data-driven predictive modeling, particularly in resource-constrained and data-scarce environments. The code used to reproduce the results will be released upon the publication.

Social Aspects · Accountability, Transparency, and Interpretability

Viet-Hung Tran, Zichi Zhang, Ngoc Doan, Xuan Hoang Nguyen, Phi Nguyen, Yimeng An, Peixin Li, Hans Vandierendonck, Ira Assent, Thai Son Mai

Deep Neural Networks (DNNs) achieve outstanding performance in Time Series Classification (TSC) yet remain opaque "black boxes", hindering their adoption in sensitive domains. While Explainable AI (XAI) aims to bridge this gap, existing TSC XAI methods rely on a single perspective and incur significant computational costs, making them difficult to integrate into real-time applications. To overcome these challenges, we propose a framework, containing two key techniques called Instance-level Multi-Expert Knowledge Distillation (IMEKD) and Semi-Amortized Optimization Explanation (SAOE). Unlike static methods, our IMEKD approach bridges the gap between explanation methods by dynamically selecting the best attribution map from a pool of "XAI experts" for each instance. We then distill this instance-optimal knowledge into a DNN using our SAOE framework, a dual-stage process that learns a global attribution via distillation and refines it using faithfulness and robustness losses, aligning the optimization with objectives of human evaluation. To the best of our knowledge, this is the first work to unify multi-expert selection with semi-amortized optimization for TSC XAI. Also, we introduce a Faithfulness-Preserving Segmentation (FPS) mechanism that converts point-wise maps into interpretable segments without sacrificing fidelity to align explanations with human intuition. Comprehensive experiments on four synthetic datasets, a ECG dataset with human-verified ground truth, and 11 multivariate UEA benchmarks across three DNN architectures show that our framework significantly outperforms the current state-of-the-art (SOTA) in terms of faithfulness, robustness, and computational efficiency.

Deep Learning · Sequential Models, Time series

Longlong Xu, Zeyan Li, Xiao He, Zhaoyang Yu, Changhua Pei, Zhe Xie, Zijun Dou, Tieying Zhang, Dan Pei

Time series forecasting plays a vital role in numerous domains. However, real-world time series are frequently contaminated by noise, missing values, and anomalies, posing significant challenges to reliable forecasting. In this work, we first systematically investigate a fundamental limitation prevalent in existing forecasting methods: an excessive reliance on the most recent observations---termed "recent-data bias". This bias renders forecasts highly vulnerable to perturbations in recent data, severely undermining prediction reliability. To address this issue, we propose TameR, a novel approach for robust time series forecasting that effectively mitigates recent-data bias via enhancing the utilization of global context. Specifically, it employs a basis-aligned randomized sampling strategy to reduce dependence on any specific recent data. Furthermore, TameR incorporates a learnable periodicity extraction module coupled with a two-stage learning protocol to robustly separate periodic patterns from the sampled residual components. Comprehensive experiments demonstrate that TameR significantly outperforms state-of-the-art methods in robustness against diverse perturbation scenarios, while achieving comparable accuracy on clean data.

Deep Learning · Sequential Models, Time series

Longlong Xu, Zeyan Li, Xiao He, Zhaoyang Yu, Dazhong Wen, Mingze Sun, Changhua Pei, Dan Pei

Time series foundation models (TSFMs) have demonstrated impressive generalization capabilities across diverse domains. However, they face significant challenges in long-term and ultra long-term forecasting. These challenges primarily arise from scalability limitations when TSFMs process extensive sequence lengths. To address this, we propose SPRINT, a training-free plug-and-play framework designed to empower TSFMs to see more, forecast better and faster during inference. The core idea is to perform forecasting in a downsampled-resolution space, enabling an extended look-back window with reduced computational costs. To avoid information loss and resolution mismatch caused by downsampling, SPRINT decomposes time series into trend and seasonal components, processing them separately. It predicts the low-frequency trend via a Resolution Interpolation workflow within the downsampled space, while preserving high-frequency details through a Pattern Replication mechanism for seasonality. Extensive experiments show that SPRINT achieves a significant improvement, increasing accuracy by 19\% while enhancing efficiency with a reduction of max memory usage by 6.4$\times$ and inference time by 16.9$\times$ compared to state-of-the-art TSFMs.

General Machine Learning · Sequential, Network, and Time Series Modeling

Wenqin Liu, Weizhi Quan, Aoqi Zuo, Erdun Gao, Vu Nguyen, Dino Sejdinovic, Howard Bondell, Mingming Gong

Data valuation quantifies the intrinsic quality of individual samples to enable principled data curation, quality control, and robust learning. For time series in critical domains such as healthcare, finance, and industrial monitoring, effective valuation methods are essential yet fundamentally lacking. Existing approaches are either model-dependent, limiting their generalizability, or designed for i.i.d. data and thus fail to capture temporal dependencies, multi-scale patterns, and non-stationary dynamics inherent to sequential data. We introduce TimeLAVA, a learning-agnostic framework that values temporal segments by their marginal contribution to minimizing distributional discrepancy between evaluated and reference data. At its core is a novel Selective Wavelet-based Wasserstein ($\mathcal{W}_\text{SW}$) discrepancy combining multi-scale wavelet transforms for temporal localization with unbalanced optimal transport for robustness to distributional shifts. Segment values are efficiently computed via sensitivity analysis without requiring model training and aggregated into point-wise scores. We provide theoretical guarantees linking valuation to model-agnostic generalization and prove bounded sensitivity to outlier contamination. Extensive experiments across anomaly detection, data pruning, and label noise detection demonstrate that TimeLAVA produces significantly more informative value scores than existing methods on diverse real-world datasets.

Applications · Time Series

Daya Kumar, Uday Devulapalli, Aarat Satsangi, Apurva Narayan

Neural operators have become a central tool in scientific machine learning for learning discretization-consistent solution operators, achieving strong results on partial differential equation (PDE) benchmarks. Physiological time series, however, are highly nonstationary and dominated by localized transient events, properties that can challenge both PDE-oriented neural operators and conventional deep models. We propose NeurOCNN, a neural-operator-based model for physiological signals that learns a function-to-label mapping while exhibiting discretization invariance. NeurOCNN integrates continuous-time, spline-parameterized convolutions with Fourier projection pooling and an attention-based task head, thereby enabling robust inference under sampling-rate shifts. Empirically, NeurOCNN outperforms standard neural-operator baselines, achieves performance comparable to state-of-the-art methods, and maintains stable accuracy under zero-shot evaluation across multiple previously unseen sampling rates. Code is available at: https://github.com/dcoder444/NeurOCNN.git

Applications · Health / Medicine

Zhengqiu Yu, Yueping Ding, Xiangrong Liu

Patient-level sepsis prediction in the ICU requires models that track how a patient’s condition evolves over time and integrate heterogeneous structured evidence from electronic health records. We present PathwayLLM, a trajectory-based framework that grounds prediction on temporal signals together with graph-structured and pathway-level clinical information derived from statistical dependency discovery. PathwayLLM follows a three-stage design. First, each observation window is encoded from multiple structured views, including physiological measurements, temporal dynamics, a heterogeneous patient–diagnosis–medication graph, and pathway signals constructed from discovered conditional independence structures among clinical variables. Second, these representations are provided to a pre-trained language model as auxiliary contextual embeddings so that risk prediction and evidence-conditioned text explanations can be learned jointly. Third, a Clinical Trajectory LSTM with Deterioration Attention aggregates window-level representations to highlight critical deterioration points and produce a patient-level risk score. On MIMIC-IV (15,410 ICU patients; 8.45% sepsis prevalence), PathwayLLM achieves AUROC 0.891 and AUPRC 0.724, outperforming strong time-series and pre-trained baselines. Ablation studies indicate that trajectory aggregation and structured clinical signals are key contributors, and clinician review suggests that the generated explanations are coherent, interpretable, and clinically relevant.

Applications · Time Series

Manrui Jiang, Jingru Huang, Yong Chen, Chen Zhang

Forecasting multivariate hidden Markov processes is challenging due to nonlinear and nonstationary observations, latent state transitions, and cross-sequence dependencies. While deep learning methods achieve strong predictive accuracy, they typically lack explicit state modeling, whereas Hidden Markov Models (HMMs) provide interpretable latent states but struggle with complex nonlinear emissions and scalability. To address these limitations, we propose DRL-STAF, a Deep Reinforcement Learning based STate-Aware Forecasting framework that jointly predicts next-step observations and estimates the corresponding hidden states for complex multivariate hidden Markov processes. Specifically, DRL-STAF models complex nonlinear emissions using deep neural networks and estimates hidden state transitions via reinforcement learning, avoiding predefined transition structures and enabling flexible adaptation to diverse and high-order dynamics. In particular, DRL-STAF remains effective when typical HMM-based methods suffer from state-space explosion. Extensive experiments demonstrate that DRL-STAF consistently outperforms HMM variants, standalone deep learning models, and existing DL–HMM hybrids in both forecasting accuracy and hidden state estimation.

Applications · Time Series

Tianmi Ma, Wenxin Huang, Jiawei Du, Lin Li, Xian Zhong, Joey Tianyi Zhou

Large Language Models (LLMs) exhibit strong capabilities in high-level semantic understanding and strategic planning, yet they suffer from persistent quantitative failure modes, such as imprecise computation and the illusion of quantitative coherence, which limit their reliability in high-stakes decision-making. To address these limitations, we decouple reasoning from computation by assigning LLMs to planning, analysis, and result interpretation, while delegating numerical computation and statistical inference to specialized external tools. These tools are not hard-coded; instead, they are created in a constrained and structured manner during planning as explicit intermediate reasoning artifacts, enabling adaptive and scenario-dependent quantitative reasoning. LLMs iteratively analyze tool outputs under diverse market conditions and leverage performance-based feedback to refine subsequent tool selection and construction, forming a bounded self-evolving loop. We instantiate this process through self-play in a controllable digital twin market, DecoupledMarket, where LLM agents continuously test, compare, and adapt their strategies. By coupling high-level planning with robust quantitative execution, the proposed framework improves the quantitative reliability of LLM-driven decision-making. Code will be released soon.

General Machine Learning · Representation Learning

Sangli Teng, Hang Liu, Koushil Sreenath

This work proves that an $n$-dimensional hybrid system can be embedded into an $m$-dimensional Euclidean space equipped with a continuous vector field on its embedded image whenever $m>2n$. This result suggests that an *intrinsically* discontinuous hybrid system generically admits a continuous *extrinsic* representation that is well-posed for differentiable optimization. Building on this existence theorem, we show that a latent Neural ODE with consistency loss in both the latent and state space can accurately recover the flow of hybrid systems. Extensive experiments suggest the proposed method outperforms the existing method in learning hybrid systems with varying geometries from only time series data.

General Machine Learning · Sequential, Network, and Time Series Modeling

Haoyi Zhou, Xin Xue, Tianyu Chen, lanhao li, Lijun SUN, Jianxin Li

The periodicity misalignment remains a challenge problem in generating time-series data across multiple domains. Existing methods model time-series interactions either at the granularity of individual points or fragmented segments. This limits their ability to capture and adapt to complex periodic patterns inherent in diverse domains. To address this, we introduce Winformer, a novel diffusion framework built on window-wise attention mechanism. We shift the fundamental processing unit in the attention mechanism from pairwise points similarity to continuous windows comparison of the entire horizon. Leveraging the adaptive window-alignment kernels derived from the frequency decomposition, Winformer brings semantically richer window representations, and effectively captures and transfers complex periodic patterns across domains. Extensive experiments on 12 real-world datasets demonstrate Winformer's effectiveness, achieving an average performance gain of 10.67% over SOTA baselines.

General Machine Learning · Sequential, Network, and Time Series Modeling

Sam Dauncey, Roger Wattenhofer

Tokenization is a hardcoded compression step which remains in the training pipeline of Large Language Models (LLMs), despite a general trend towards architectures becoming increasingly end-to-end. Prior work has shown promising results at scale in bringing this compression step inside the LLMs' architecture with heuristics to draw token boundaries, and also attempts to learn these token boundaries with straight-through estimates, which treat the problem of drawing discrete token boundaries as a continuous one. We show that these token boundaries can instead be learned using score function estimates, which have tighter theoretical guarantees due to directly optimizing the problem of drawing discrete token boundaries to minimize loss. We observe that techniques from reinforcement learning, such as time discounting, are necessary to reduce the variance of this score function sufficiently to make it practicable. We demonstrate that the resultant method outperforms prior proposed straight-through estimates, both qualitatively and quantitatively at the $100$ million parameter scale.

Deep Learning · Generative Models and Autoencoders

Zhiyuan Ouyang, Simei Huang, Zuokai Wen, Xiangyun Zhang, Junchi Yan

A probability distribution $\mathbb{P}$ is a measure on a $\sigma$-algebra, assigning mass to sets rather than points. This poses a challenge for the training of neural networks, which often struggle to reconstruct the global topology of continuous manifolds from sparse samples. We mitigate this issue by innovatively transforming $\mathbb{P}$ into a continuous function via spectral methods, providing theoretical guarantees for the convergence of the learned distribution to the true distribution. Specifically, we introduce a network, PriSpecNet, with a single-function evaluation (1-NFE) Pseudoinverse Lévy Inversion (PiLI) solver that regards generation as a fast analytical problem, eliminating the need for iterative numerical integration. By reformulating the generation in the spectral domain, we bypass the computationally expensive sampling trajectory while maintaining full compatibility with the stochastic interpolants. We test our PriSpecNet in two applications. For time series, it unifies generation and forecasting, outperforming state-of-the-art (SOTA) baselines with Context-FID reductions of 50.0\%, 41.5\%, 80.6\%, and 63.1\% on Sines, Solar, ETTh, and Stock benchmarks, respectively, also decreasing forecasting MSE by 29.8\% on Solar and 23.8\% on Stock. For vision on ImageNet $256 \times 256$, 1-NFE PiLI achieves a competitive FID of **1.66** using only **26** Gflops, representing a **170** $\times$ reduction in total Gflops compared to the 4,436 Gflops required by the advanced 25-NFE DPM-Solver++.

Applications · Time Series

Jiafeng Lin, Yuxuan Wang, HUAKUN LUO, Jianmin Wang, Zhongyi Pei

Multimodal time series forecasting has garnered significant attention for its potential to provide more robust and accurate predictions than traditional single-modality models by leveraging rich information inherent in other modalities. However, due to fundamental challenges in modality alignment, existing methods often struggle to effectively incorporate multimodal data into predictions, particularly textual information that has a causal influence on time series fluctuations, such as emergency reports and policy announcements. In this paper, we reflect on the role of textual information in numerical forecasting and propose **Ti**me series transformers with Multimodal **Mi**xture-of-Experts, **TiMi**, to unleash the causal reasoning capabilities of LLMs. Concretely, TiMi utilizes language models to generate inferences on future developments, which then serve as guidance for time series forecasting. To seamlessly integrate both exogenous factors and time series into predictions, we introduce a Multimodal Mixture-of-Experts (MMoE) module as a lightweight plug-in to empower Transformer-based time series models for multimodal forecasting, eliminating the need for explicit representation-level alignment. Experimentally, our proposed TiMi demonstrates consistent state-of-the-art performance on sixteen real-world multimodal forecasting benchmarks, outperforming advanced baselines while offering strong adaptability and interpretability.

Applications · Time Series

Austin Feng, Andreas Varvarigos, Ioannis Panitsas, Daniela Fernandez, Yuwei Guo, Jinbiao Wei, Chen, Ali Maatouk, Leandros Tassiulas, ZHITAO YING

Modern enterprises generate vast streams of time series metrics when monitoring complex systems, known as observability data. Unlike conventional time series from domains such as climate, observability data are zero-inflated, highly stochastic, and exhibit minimal temporal structure. Despite their importance, observability datasets are underrepresented in public benchmarks due to proprietary restrictions. Existing datasets are often anonymized and normalized, removing scale information and limiting their use for tasks such as anomaly detection, root-cause analysis, and multi-modal reasoning. To address this gap, we introduce TelecomTS, a large-scale observability dataset derived from a 5G telecommunications network. TelecomTS features heterogeneous, de-anonymized covariates with explicit scale information and provides a suite of downstream tasks, including anomaly detection, root-cause analysis, and multi-modal question-answering. Benchmarking state-of-the-art time series, language, reasoning, and multi-modal models reveals that existing approaches struggle with the abrupt, noisy, and high-variance dynamics of observability data. Our experiments also underscore the importance of preserving covariates’ absolute scale, emphasizing the need for foundation time series models that natively leverage scale information for practical observability applications. The code is available at: \url{https://anonymous.4open.science/r/TelecomTS_Benchmark-72AF}.