Generative latent diffusion models (LDMs) have been extensively applied in various fields yet underperform in time-series prediction. Therefore, We propose the Re-Diffusion model, a latent diffusion approach that generates backbone residuals specifically tailored for time-series forecasting. The model comprises a variational autoencoder that compresses the residuals between the actual future values and the predictions from the backbone into latent space. It also includes a conditional diffusion generator to forecast the potential distribution of these residuals. Our findings reveal that this latent-space methodology particularly enhances existing backbone predictors, by effectively reducing prediction bias through an advanced estimation of complex error distributions. While previous diffusion-based models tend to struggle with long-term forecasting, Re-Diffusion integrates the strengths of diffusion methods, leading to improvements in long-term predictions. Our experimental results indicate that the Re-Diffusion model achieves a 10% promotion over state-of-art predictors, marking a significant advancement in the field of time-series forecasting.
论文检索
输入标题、作者或关键词，从 954 篇学术成果中精准定位
Knowledge Tracing (KT) is a core task in intelligent tutoring systems, designed to model the dynamic evolution of students' knowledge states by predicting their performance on specific problems. However, existing KT models, primarily developed for traditional subject domains, exhibit poor generalization to programming tasks due to several distinct challenges. First, student responses in programming tasks are characterized by high uncertainty. Second, programming tasks typically provide continuous scores based on the proportion of passed test cases, in contrast to the binary correctness assumed by traditional KT models, with these test-case-based scores often being noisy. Third, most KT models rely exclusively on the understanding of knowledge concepts for performance prediction, whereas success in programming tasks is heavily contingent upon logical reasoning and problem-specific comprehension. To address these issues, we propose an LLM-driven Interaction Enrichment Framework (MIE) to mitigate high uncertainty and problematic labeling, and introduce the Multi-Level Programming Knowledge Tracing (MLPKT) model to capture students' knowledge states across multiple dimensions. MLPKT conducts multi-layer analysis of student submissions to identify the root causes of errors and assign semantically meaningful fine-grained labels. Additionally, we propose a three-level, three-phase KT architecture that captures knowledge dynamics across three dimensions—problems, concepts, and logical skills—through the phases of learning, forgetting/reinforcement, and application. Extensive experiments on three datasets demonstrate that MIE+MLPKT consistently outperforms 18 baseline methods. Our code is available at: https://anonymous.4open.science/r/MIE-MLPKT-D654.
As an important task in modern web technologies, multivariate time series forecasting drives many core functionalities. However, in realistic web-scale environments, sensor failures, privacy filtering, sampling, and instrumentation churn frequently yield missing variables, making training sets complete while test sets contain only a small subset of variables. The challenge lies in utilizing incomplete data for forecasting, which is known as Variable Subset Forecasting (VSF). Distribution shift is inherent to time series and remains in VSF, including inter-series shift as changes in cross-series correlations and intra-series shift as substantial distribution differences within the same series across different time windows. Existing VSF approaches typically impute missing variables and then forecast on the completed series, yet they overlook these shifts and thus underperform in realistic web-scale scenarios. To address these challenges, we propose Shift Resilient Diffusive Imputation (SRDI), a framework tailored to VSF and robust to distribution shift. Specifically, SRDI integrates a divide-conquer strategy with the denoising process, which decomposes the input into invariant patterns and variant patterns, representing the temporally stable parts of inter-series correlation and the highly fluctuating parts, respectively. By extracting spatiotemporal features from each part separately and then appropriately combining them, inter-series shift can be effectively mitigated. Then, we innovatively organize SRDI and the forecasting model into a meta-learning paradigm tailored for VSF scenarios. We address the intra-series shift by treating time windows as tasks during training and employing an adaptation process before testing, which naturally supports robust online forecasting in dynamic web environments. Extensive experiments on four datasets have demonstrated our superior performance compared with state-of-the-art methods. Our code is available at https://github.com/xhhmacau/SRDI.
The strategic planning and reliability of modern web services, from cloud infrastructures to e-commerce platforms, increasingly hinge on accurate long-term forecasting of high-dimensional time series. A fundamental challenge within this task is modeling the intricate point-to-point dependencies that span across both time and variable dimensions. However, many existing methods face restricted direction modeling and computational inefficiency due to their reliance on localized paradigms and Transformer architectures. To address these, we propose replacing Self-Attention with autocorrelation, achieving two key innovations: 1) We propose calculating autocorrelation across both variable and time dimensions, which is a global paradigm, to model point-to-point dependencies. 2) Our proposed Spectral Product Mechanism (SPM) optimizes traditional autocorrelation into a data-driven form suitable for deep learning. Moreover, SPM reformulates autocorrelation as spectral product and reduces the complexity from O(N2) to O(NlogN), while its Hadamard product-based correlation score matrix further reduces core computation to O(N) compared to Self-Attention's O(N2) matrix multiplication. We further propose a Generalized Spectral Product Mechanism (GSPM), which extends traditional autocorrelation by mapping input into distinct feature representations, enabling modeling of complex dependencies through cross-feature correlations. SPM and GSPM surpass current state-of-the-art (SOTA) methods on 14 authoritative benchmarks, collectively securing the top rank on 22 out of 28 metrics, while ranking 1st in time, 2nd in memory, and 2nd in parameter overhead. Source code is available at: https://github.com/lxy-PhD2022/SPM.
Low-rank adaptation (LoRA) has emerged as an efficient fine-tuning technique for large language models, enabling parameter-efficient updates while maintaining task performance. However, LoRA suffers from two key issues: 1) inefficient feature learning when the width n (embedding dimension) is large, and 2) ineffective updates to the adapter matrix A due to the initialization of B as zero. We propose LoRA-E2, which utilizes a Gaussian initialization with variance Θ(n-3/4 ) for A, and employs the Gauss-Seidel iteration to train B and A. We theoretically show that LoRA-E2 enables more stable and efficient feature learning with effective parameter updates over standard LoRA. Empirically, LoRA-E2 achieves consistent gains in both natural language understanding and generation tasks. On the GLUE benchmark with T5-base, it improves performance by 1–10% over LoRA. When fine-tuning LLaMA 2-7B on MetaMathQA with GSM8K as validation, LoRA-E2 surpasses LoRA by 1–2% and converges up to ∼3× faster. Code is available at https://github.com/whu-totemdb/LoRA-E2.
Web-scale content is rich in modalities yet frequently incomplete due to device limits, transmission errors, or privacy controls, making learning with missing modalities a core challenge. Prior reconstruction and alignment strategies often fail to preserve a stable class geometry when inputs are partial, leading to anchor drift -- a shift of class prototypes between complete and incomplete views that distorts the shared representation space and degrades generalization. We introduce HiCoD (Hierarchical Consistency-Guided Pro mpt Distillation), which learns a robust, class-anchored semantic space. HiCoD combines: (1) a modality-aware semantic graph that restores cross-modal structure under partial observations; (2) dual-level anchoring that unifies large-language-model–derived global category prototypes with top-K local exemplars to balance cross-modal coherence and modality-specific detail; and (3) multi-level distillation that aligns unimodal features, fused embeddings, and prompt-completed signals within a single anchor space. Across CMU-MOSI, CMU-MOSEI, and additional benchmarks, HiCoD sets a new state of the art under both fixed-pattern and random missingness, improving Acc-2 by up to 6.4 points over MPLMM and remaining robust when key modalities are absent.
The prevalence of sarcasm in multimodal dialogues on the social platforms presents a crucial yet challenging task for understanding the true intent behind online content. Comprehensive sarcasm analysis requires two key aspects: Multimodal Sarcasm Detection (MSD) and Multimodal Sarcasm Explanation (MuSE). Intuitively, the act of detection is the result of the reasoning process that explains the sarcasm. Current research predominantly focuses on addressing either MSD or MuSE as a single task. Even though some recent work has attempted to integrate these tasks, their inherent causal dependency is often overlooked. To bridge this gap, we propose MuVaC, a variational causal inference framework that mimics human cognitive mechanisms for understanding sarcasm, enabling robust multimodal feature learning to jointly optimize MSD and MuSE. Specifically, we first model MSD and MuSE from the perspective of structural causal models, establishing variational causal pathways to define the objectives for joint optimization. Next, we design an alignment-then-fusion approach to integrate multimodal features, providing robust fusion representations for sarcasm detection and explanation generation. Finally, we enhance the reasoning trustworthiness by ensuring consistency between detection results and explanations. Experimental results demonstrate the superiority of MuVaC in public datasets, offering a new perspective for understanding multimodal sarcasm.
Traffic prediction plays a pivotal role in contemporary web technologies, motivating various intelligent web services such as route planning and remote traffic management. Many recent proposals that target deep learning for traffic prediction solely leverage historical traffic observations to predict future ones. However, traffic prediction is always susceptible to different factors such as road networks and social events, exhibiting different modalities. Most existing methods focus on a single modality, failing to capture the comprehensive traffic patterns among various factors, resulting in sub-optimal performance. Web-sourced geo-images, e.g., satellite imagery, encompass comprehensive contextual information and offer an effective way to represent diverse modalities. To unleash the power of such geo-images, we propose VisionST, a Vision-augmented Spatial-Temporal Neural Network, which coordinates cross-modal traffic prediction with interactive geo-image encoding. To bolster resilience against highly intricate and overlapping traffic patterns, VisionST features a visual semantic extraction mechanism and a pattern-guided aggregation mechanism. The former extracts node-level visual tokens and node-to-node visual relation patterns from geo-referenced images. The latter generates relation patterns that encompass visual, spatial, and temporal aspects, constraining nodes to interact with these relation patterns for contextual information interaction. Extensive experiments on real large-scale datasets offer insight into the effectiveness of the proposed solutions, showing that VisionST consistently outperforms state-of-the-art baselines.
Time series anomaly detection aims to identify samples that deviate from a normal sample distribution in a time series, enabling various web-centric applications. Most existing approaches are static, targeting pre-defined types of anomalies. These methods thus fail to work well on streaming time series with changing data distributions and anomaly formats. To contend with such streaming time series and to accommodate memory constraints, we propose the first data-efficient streaming time series anomaly detection framework, called DESS. To accumulate historical knowledge, DESS includes a novel evolving proxy generation module to synthesize a small but informative proxy summarizing the historical data, facilitating data efficiency. Next, DESS employs an innovative heterogeneous temporal feature extraction module to explicitly capture correlations of multi-level time series semantics. Finally, DESS enables fast streaming anomaly detection by employing a parameter-efficient training scheme that only activates a subset of lightweight parameters while ensuring performance. Extensive experiments on real data offer insight into the effectiveness and efficiency of DESS, showing that it is able to outperform the best baselines by up to 17.53% while reducing the training time by up to 64.88%.
Towards Multi-Label Text Interpretation with Chain-of-Thought Prompting and Contextualized Knowledge
Existing multi-label topic models face several challenges when interpreting texts annotated with multiple labels: (1) they often associate irrelevant text segments with incorrect labels, which negatively impacts both segment and label interpretation; (2) they fail to effectively capture the semantic relationships between tokens and labels within the segment; (3) they do not integrate contextualized knowledge that could improve interpretability. To overcome these issues, we introduce the Contextualized Prompting Topic Model (CPTM). CPTM utilizes Chain-of-Thought (CoT) prompting to better align text segments with their semantically relevant labels. Furthermore, it integrates label-specific token visualization and topic mining procedure to facilitate the interpretation of tokens and labels. Experimental evaluations conducted on three multi-label text datasets show that CPTM significantly outperforms existing models in both segment and label interpretation. Human assessments also verify CPTM's effectiveness in accurately identifying label-relevant tokens within segments and providing insightful token-level interpretation.
Transformer-based foundation models have achieved remarkable progress in tasks such as time-series forecasting and image segmentation. However, they frequently suffer from error accumulation in multivariate long-sequence prediction and exhibit vulnerability to out-of-distribution samples in image-related tasks. Furthermore, these challenges become particularly pronounced in large-scale Web data analysis tasks, which typically involve complex temporal patterns and multimodal features. This complexity substantially increases optimization difficulty, rendering models prone to stagnation at saddle points within high-dimensional parameter spaces. To address these issues, we propose a lightweight Transformer architecture in conjunction with a novel Escape-Explore Optimizer (EEO). The optimizer enhances both exploration and generalization while effectively avoiding sharp minima and saddle-point traps. Experimental results show that, in representative Web data scenarios, our method achieves performance on par with state-of-the-art models across 11 time-series benchmark datasets and the Synapse medical image segmentation task. Moreover, it demonstrates superior generalization and stability, thereby validating its potential as a versatile cross-task foundation model for Web-scale data mining and analysis.
Recent advances in generative artificial intelligence, particularly large language models (LLMs), have opened new opportunities for enhancing recommender systems (RecSys). Most existing LLM-based RecSys approaches operate in a discrete space, using vector-quantized tokenizers to align with the inherent discrete nature of language models. However, these quantization methods often result in lossy tokenization and suboptimal learning, primarily due to inaccurate gradient propagation caused by the non-differentiable argmin operation in standard vector quantization. Inspired by the emerging trend of embracing continuous tokens in language models, we propose ContRec, a novel framework that seamlessly integrates continuous tokens into LLM-based RecSys. Specifically, ContRec consists of two key modules: a σ-VAE Tokenizer, which encodes users/items with continuous tokens; and a Dispersive Diffusion module, which captures implicit user preference. The tokenizer is trained with a continuous Variational Auto-Encoder (VAE) objective, where three effective techniques are adopted to avoid representation collapse. By conditioning on the previously generated tokens of the LLM backbone during user modeling, the Dispersive Diffusion module performs a conditional diffusion process with a novel Dispersive Loss, enabling high-quality user preference generation through next-token diffusion. Finally, ContRec leverages both the textual reasoning output from the LLM and the latent representations produced by the diffusion model for Top-K item retrieval, thereby delivering comprehensive recommendation results. Extensive experiments on four datasets demonstrate that ContRec consistently outperforms both traditional and state-of-the-art LLM-based recommender systems. Our results highlight the potential of continuous tokenization and generative modeling for advancing the next generation of recommender systems.
Dynamic Facial Expression Recognition (DFER), as a crucial part of affective computing, has broad applications in many areas such as human-computer interaction and social media content analysis. Effectively integrating multimodal information, particularly audio-visual signals, remains the core challenge. However, existing approaches are generally constrained by two major challenges: (1) shallow and static fusion mechanisms, which fail to capture the dynamic co-evolution of audio-visual features during deep interaction; (2) implicit and coarse alignment strategies, which are insufficient to bridge the modality gap caused by heterogeneous feature distributions. To address these issues, we propose a novel framework, BHGap, which integrates deep iterative prompt generation with multi-stage feature alignment and fusion. The key idea is to reformulate audio-visual collaboration from a one-shot fusion event into a continuous, reciprocal generation process that spans every layer of frozen backbone encoders. Specifically, we design a State Space Model (SSM)-based cross-modal prompt generator that dynamically produces ''guidance prompts'' for the counterpart modality at each encoding layer, thereby enabling deep and fine-grained feature co-evolution. Beyond encoding, we further introduce a coarse-to-fine multi-stage alignment module: at the macro level, low-rank adversarial alignment is employed to establish spatio-temporal congruity between audio and video while reducing global distributional discrepancies; at the micro level, Maximum Mean Discrepancy (MMD) constraints combined with implicit differentiation optimization ensure fine-grained statistical consistency and semantic alignment. Extensive experiments on the public DFEW and MAFW datasets demonstrate that our method achieves state-of-the-art performance, offering a new paradigm of deep iterative fusion and explicit alignment for multimodal emotion recognition. Code is available at https://github.com/NDYZD666/-public-BHGap.
Insider threat detection (ITD) is notoriously difficult: malicious actions are rare, context-dependent, and deliberately hidden within massive volumes of legitimate user behavior. Existing ITD methods rely on single- or fused-view models, which lack extensibility and therefore fail to leverage the supervisory signals from newly introduced complementary views. While ensembling is a natural next step, its direct application to ITD confronts three core obstacles: scalability bottlenecks from independently trained sub - models, semantic misalignment across heterogeneous feature spaces, and view imbalance, where strong views overshadow weaker yet informative ones. In this work, we propose Insight-LLM, the first extensible multi-view fusion framework tailored for ITD. Insight-LLM encodes each view with frozen pre-trained backbones and aligns heterogeneous representations into a unified semantic space via a lightweight ViewAdapter, enabling coherent cross-view reasoning without incurring additional training overhead. A context-adaptive fusion module dynamically re-weights views to emphasize subtle yet semantically consistent threat signals, and the fused representation is integrated with task prompts for lightweight LLM fine-tuning. Experiments on CERT datasets show that Insight-LLM improves F1 by up to 4.8% and reduces false positives by 61%, while decreasing training time per newly added view by up to 83.2% compared with the simple Ensemble method.
Time series forecasting is crucial for the development of sophisticated web technologies, driving smarter, more responsive, and data-driven web applications. A key to accurate forecasting lies in effectively capturing the intricate dependencies among different variables (channels). While existing channel-dependent methods have shown strong performance by explicitly modeling inter-channel relationships, they face two critical challenges when applied to high-dimensional datasets with thousands of channels. First, the computational complexity of them grows quadratically with the number of channels, leading to significant scalability issues. Second, attention weights reveal that inter-channel dependencies exhibit both local clusters and global structures, yet current methods fail to disentangle these heterogeneous patterns, resulting in mutual interference and degraded forecasting accuracy. To address these challenges, we propose a novel Channel Reordering-Aligned group Fusion Transformer (CRAFT) for high-dimensional time series forecasting. Specifically, we design an energy-based channel reordering mechanism that reorganizes channels into a minimal-energy state, preserving inherent local-global structures. Building on reordered structure, we introduce a group fusion Transformer that explicitly separates local and global dependencies, significantly reducing computational complexity while enhancing representational clarity. Experiments on high-dimensional datasets demonstrate that CRAFT consistently outperforms baselines, achieving higher forecasting accuracy with lower computational overhead.
The World Wide Web needs reliable predictive capabilities to respond to changes in user behavior and usage patterns. Time series forecasting (TSF) is a key means to achieve this goal. In recent years, the large language models (LLMs) for TSF (LLM4TSF) have achieved good performance. However, there is a significant difference between pretraining corpora and time series data, making it hard to guarantee forecasting quality when directly applying LLMs to TSF; fine-tuning LLMs can mitigate this issue, but often incurs substantial computational overhead. Thus, LLM4TSF faces a dual challenge of prediction performance and compute overhead. To address this, we aim to explore a method for improving the forecasting performance of LLM4TSF while freezing all LLM parameters to reduce computational overhead. Inspired by in-context learning (ICL), we propose LVICL. LVICL uses our vector-injected ICL to inject example information into a frozen LLM, eliciting its in-context learning ability and thereby enhancing its performance on the example-related task (i.e., TSF). Specifically, we first use the LLM together with a learnable context vector adapter to extract a context vector from multiple examples adaptively. This vector contains compressed, example-related information. Subsequently, during the forward pass, we inject this vector into every layer of the LLM to improve forecasting performance. Compared with conventional ICL that adds examples into the prompt, our vector-injected ICL does not increase prompt length; moreover, adaptively deriving a context vector from examples suppresses components harmful to forecasting, thereby improving model performance. Extensive experiments demonstrate the effectiveness of our approach.
The World Wide Web thrives on intelligent services that rely on accurate time series classification, which has recently witnessed significant progress driven by advances in deep learning. However, existing studies face challenges in domain incremental learning. In this paper, we propose a lightweight and robust dual-causal disentanglement framework (DualCD) to enhance the robustness of models under domain incremental scenarios, which can be seamlessly integrated into time series classification models. Specifically, DualCD first introduces a temporal feature disentanglement module to capture class-causal features and spurious features. The causal features can offer sufficient predictive power to support the classifier in domain incremental learning settings. To accurately capture these causal features, we further design a dual-causal intervention mechanism to eliminate the influence of both intra-class and inter-class confounding features. This mechanism constructs variant samples by combining the current class's causal features with intra-class spurious features and with causal features from other classes. The causal intervention loss encourages the model to accurately predict the labels of these variant samples based solely on the causal features. Extensive experiments on multiple datasets and models demonstrate that DualCD effectively improves performance in domain incremental scenarios. We summarize our rich experiments into a comprehensive benchmark to facilitate research in domain incremental time series classification.
Table reasoning with large language models (LLMs) plays a critical role in building intelligent systems capable of understanding and analyzing tabular data. Despite recent progress, existing methods still face key limitations: their reasoning processes lacks depth and explicit multi-step reasoning, often relying solely on implicit language model understanding. In addition, their reasoning processes suffer from instability, primarily caused by model uncertainty. In this work, we propose STaR, a novel slow-thinking model that can achieve effective and stable table reasoning. To enable effective multi-step reasoning, we design a two-stage training framework consisting of supervised fine-tuning (SFT) warm-up followed by reinforced fine-tuning (RFT). Specifically, in the SFT stage, we construct a high-quality dataset through automatic self-verification. In the RFT stage, we introduce a difficulty-aware reinforcement learning mechanism to further enhance reasoning capabilities. Furthermore, to improve reasoning stability, we introduce trajectory-level uncertainty quantification, which fuses token-level confidence with answer-level consistency, enabling the selection of better reasoning trajectories. Extensive experiments demonstrate that STaR-8B achieves state-of-the-art performance on in-domain benchmarks and exhibits strong generalization to out-of-domain datasets, highlighting its potential for enhancing both effectiveness and stability in table reasoning.
The proliferation of misinformation on video-sharing platforms demands robust detection of video fake news. Existing methods struggle to integrate external world knowledge with internal multimodal cues, limiting their generalization and robustness. In this work, we propose TrueLens, a new framework for video fake news detection that gathers and consolidates dual-level evidence \zznotethrough three primary components, \ie, External Precedent Retriever, Adversarial Contrastor, and Internal Evidential Logic Fusion. At the external level, the External Precedent Retriever first decomposes the query video into textual, visual, and audio queries while leveraging multimodal large language models (MLLMs) to enhance the overall semantic representation. It then applies an entropy-guided multimodal retrieval mechanism to identify the two most similar reference videos from a gallery of real and fake samples, \ie, one real and one fake video. The Adversarial Contrastor integrates these references with the input video through contrastive attention, enhancing contextual reasoning. At the internal level, our Evidential Logic Fusion module aggregates multimodal signals from the Adversarial Contrastor to produce consistent, robust predictions. Extensive experiments on three benchmarks show that the proposed TrueLens consistently surpasses competitive baselines under both temporal and event settings by a clear margin, yielding up to a +21.60% F1 improvement under the event setting and achieving 93.73%, 90.64%, and 98.83% accuracy on the FakeSV, FakeTT, and FVC datasets under the temporal setting. The code for our project is available at https://github.com/JunyiChen-ai/TrueLens.
With the increasing relevance of web mining and content analysis in uncovering mobility patterns from large-scale online data, traffic flow prediction plays a crucial role in proactive urban planning and enhancing the responsiveness of intelligent transportation systems. However, existing traffic flow prediction methods often fail to explicitly capture correlations in continuous multivariate sequences that are naturally suited for trend and periodic pattern extraction by vision models and rely on fixed spatial graphs that neglect the cognitive advantages of granular-ball structures, which limits their ability to model interactions and strengthen spatiotemporal dependencies. To address these challenges, this paper proposes a Cross-domain Mamba framework that integrates Multi-scale Imaging and Granular-ball Computing for traffic flow prediction (MIGC-CMamba). First, a multi-scale sequence imaging method is presented, which converts the original time series into image modality and leverages MambaVision to capture both local and global dependencies. Second, a multi-granularity spatial graph is constructed via granular-ball clustering, which balances global trend representation and local detail preservation. Third, a cross-domain enhancement mechanism adaptively integrates temporal and spatial domains, strengthening spatiotemporal dependencies. Lastly, extensive experiments demonstrate superior performance over state-of-the-art baselines, highlighting how vision-based imaging, cognition-inspired granular-ball modeling, and content-aware mining jointly advance the modeling of spatiotemporal dependencies in traffic flow prediction.