Empowering large language models (LLMs) with effective tool utilization capabilities is crucial for enabling AI agents to solve complex problems. However, current models face two major limitations: (1) unreliable tool planning and invocation due to low-quality instruction datasets (e.g., widespread hallucinated API calls), and (2) weak tool reflection abilities (over 90% of errors cannot be corrected) resulting from static imitation learning. To address these critical limitations, we propose Tool-MVR, a novel Tool-Augmented LLM that achieves comprehensive System 2 reasoning through two key innovations. Specifically, we first introduce Multi-Agent Meta-Verification (MAMV), a systematic pipeline that rigorously validates APIs, queries, and reasoning trajectories to construct ToolBench-V, a new high-quality instruction dataset that addresses the limitation of unreliable tool planning and invocation. Second, we propose Exploration-based Reflection Learning (EXPLORE), which enhances tool reflection capabilities by leveraging tool feedback through a dynamic "Error → Reflection → Correction" learning paradigm, resulting in our reflection dataset ToolBench-R and addressing the critical weakness in tool reflection. Finally, we obtain Tool-MVR by finetuning open-source LLMs (e.g., Qwen-7B) on both ToolBench-V and ToolBench-R. Our experiments demonstrate that Tool-MVR achieves state-of-the-art performance on StableToolBench, surpassing both ToolLLM (by 23.9%) and GPT-4 (by 15.3%) while reducing API calls by 31.4%, with strong generalization capabilities across unseen tools and scenarios. Additionally, on our proposed RefineToolBench, the first benchmark specifically designed to evaluate tool reflection capabilities. Tool-MVR achieves a 58.9% error correction rate, significantly outperforming ToolLLM's 9.1%.
论文检索
输入标题、作者或关键词，从 12,319 篇学术成果中精准定位
Beyond Fixed Variables: Expanding-variate Time Series Forecasting via Flat Scheme and Spatio-temporal Focal Learning
PDF ↗Multivariate Time Series Forecasting (MTSF) has long been a key research focus. Traditionally, these studies assume a fixed number of variables, but in real-world applications, Cyber-Physical Systems often expand as new sensors are deployed, increasing variables in MTSF. In light of this, we introduce a novel task, Expanding-variate Time Series Forecasting (EVTSF). This task presents unique challenges, specifically (1) handling inconsistent data shapes caused by adding new variables, and (2) addressing imbalanced spatio-temporal learning, where expanding variables have limited observed data due to the necessity for timely operation. To address these challenges, we propose STEV, a flexible spatio-temporal forecasting framework. STEV includes a new Flat Scheme to tackle the inconsistent data shape issue, which extends the graph-based spatio-temporal modeling architecture into 1D space by flattening the 2D samples along the variable dimension, making the model variable-scale-agnostic while still preserving dynamic spatial correlations through a holistic graph. Additionally, we introduce a novel Spatio-temporal Focal Learning strategy that incorporates a negative filter to resolve potential conflicts between contrastive learning and graph representation, and a focal contrastive loss as its core to guide the framework to focus on optimizing the expanding variables. To evaluate the effectiveness of STEV, we benchmark EVTSF performance on three real-world datasets from various domains and compare it against three potential solutions employing state-of-the-art (SOTA) MTSF models tailored for EVSTF. Experimental results show that STEV significantly outperforms its competitors, especially in handling expanding variables. Notably, STEV, with only 5% of observations during the expanding period, is on par with SOTA MTSF models trained with complete data. Further exploration of various expanding scenarios underscores the generalizability of STEV in real-world applications.
The remarkable prompting capability of large language models (LLMs) offers substantial convenience to users across diverse backgrounds. Nevertheless, as the sensitive information within prompts is inevitably exposed to LLMs, caution must be exercised to preserve privacy. Among various studies, text anonymization is considered an effective approach to preventing privacy leakage in prompts through text substitution. However, existing works overemphasize privacy while overlooks preserving contextual integrity, degrading semantic consistency. To address these concerns, this paper introduces a context-sensitive prompt privacy-preserving framework, namely Adaptive Linguistic Sanitization and Anonymization (ALSA). In specific, ALSA incorporates a three-dimensional scoring mechanism to dynamically quantify the substitutability of each word within a prompt by integrating the Privacy Leakage Risk Score (PLRS), the Contextual Information Importance Score (CIIS), and the Task Relevance Score (TRS). Subsequently, a clustering technique is adopted to dynamically determine the threshold for assigning an anonymization action (i.e., Retain, Replace, Encrypt, or Delete) by balancing privacy, semantics, and task relevance. Extensive experiments on five benchmark datasets validate the superiority of ALSA over state-of-the-art baselines in terms of accuracy, privacy preservation, and semantic integrity.
Survival analysis, which estimates the probability of event occurrence over time from censored data, is fundamental in numerous real-world applications, particularly in high-stakes domains such as healthcare and risk assessment. Despite advances in numerous survival models, quantifying the uncertainty of predictions from these models remains underexplored and challenging. The lack of reliable uncertainty quantification limits the interpretability and trustworthiness of survival models, hindering their adoption in clinical decision-making and other sensitive applications. To bridge this gap, in this work, we introduce SurvUnc, a novel meta-model based framework for post-hoc uncertainty quantification for survival models. SurvUnc introduces an anchor-based learning strategy that integrates concordance knowledge into meta-model optimization, leveraging pairwise ranking performance to estimate uncertainty effectively. Notably, our framework is model-agnostic, ensuring compatibility with any survival model without requiring modifications to its architecture or access to its internal parameters. Especially, we design a comprehensive evaluation pipeline tailored to this critical yet overlooked problem. Through extensive experiments on four publicly available benchmarking datasets and five representative survival models, we demonstrate the superiority of SurvUnc across multiple evaluation scenarios, including selective prediction, misprediction detection, and out-of-domain detection. Our results highlight the effectiveness of SurvUnc in enhancing model interpretability and reliability, paving the way for more trustworthy survival predictions in real-world
Cell fate decisions are highly coordinated processes governed by complex interactions among numerous regulatory genes, while disruptions in these mechanisms can lead to developmental abnormalities and disease. Traditional methods often fail to capture such combinatorial interactions, limiting their ability to fully model cell fate dynamics. Here, we introduce MetaVelo, a global feature explanation framework for identifying key regulatory gene sets influencing cell fate transitions. MetaVelo models these transitions as a black-box function and employs a differentiable neural ordinary differential equation (ODE) surrogate to enable efficient optimization. By reparameterizing the problem as a controllable data generation process, MetaVelo overcomes the challenges posed by the non-differentiable nature of cell fate dynamics. Benchmarking across diverse stand-alone and longitudinal single-cell RNA-seq datasets and three black-box cell fate models demonstrates its superiority over 12 baseline methods in predicting developmental trajectories and identifying combinatorial regulatory gene sets. MetaVelo further distinguishes independent from synergistic regulatory genes, offering novel insights into the gene interactions governing cell fate. With the growing availability of high-resolution single-cell data, MetaVelo provides a scalable and effective framework for advancing developmental biology and therapeutic applications.
Recently, Graph Neural Networks (GNNs) have become the dominant approach for Knowledge Graph-aware Recommender Systems (KGRSs) due to their proven effectiveness. Building upon GNN-based KGRSs, Self-Supervised Learning (SSL) has been incorporated to address the sparity issue, leading to longer training time. However, through extensive experiments, we reveal that: (1)compared to other KGRSs, the existing GNN-based KGRSs fail to keep their superior performance under sparse interactions even with SSL. (2) More complex models tend to perform worse in sparse interaction scenarios and complex mechanisms, like attention mechanism, can be detrimental as they often increase learning difficulty. Inspired by these findings, we propose LightKG, a simple yet powerful GNN-based KGRS to address sparsity issues. LightKG includes a simplified GNN layer that encodes directed relations as scalar pairs rather than dense embeddings and employs a linear aggregation framework, greatly reducing the complexity of GNNs. Additionally, LightKG incorporates an efficient contrastive layer to implement SSL. It directly minimizes the node similarity in original graph, avoiding the time-consuming subgraph generation and comparison required in previous SSL methods. Experiments on four benchmark datasets show that LightKG outperforms 12 competitive KGRSs in both sparse and dense scenarios while significantly reducing training time. Specifically, it surpasses the best baselines by an average of 5.8% in recommendation accuracy and saves 84.3% of training time compared to KGRSs with SSL. Our code is available at https://github.com/1371149/LightKG.
Adapting Pretrained Language Models for Citation Classification via Self-Supervised Contrastive Learning
PDF ↗Citation classification, which identifies the intention behind academic citations, is pivotal for scholarly analysis. Previous works suggest fine-tuning pretrained language models (PLMs) on citation classification datasets, reaping the reward of the linguistic knowledge they gained during pretraining. However, directly fine-tuning for citation classification is challenging due to labeled data scarcity, contextual noise, and spurious keyphrase correlations. In this paper, we present a novel framework, Citss, that adapts the PLMs to overcome these challenges. Citss introduces self-supervised contrastive learning to alleviate data scarcity, and is equipped with two specialized strategies to obtain the contrastive pairs: sentence-level cropping, which enhances focus on target citations within long contexts, and keyphrase perturbation, which mitigates reliance on specific keyphrases. Compared with previous works that are only designed for encoder-based PLMs, Citss is carefully developed to be compatible with both encoder-based PLMs and decoder-based LLMs, to embrace the benefits of enlarged pretraining. Experiments with three benchmark datasets with both encoder-based PLMs and decoder-based LLMs demonstrate our superiority compared to the previous state of the art. Our code is available at: github.com/LITONG99/Citss
A significant challenge for Graph Neural Networks (GNNs) lies in accurately performing node classification tasks on heterophilous graphs. Traditional GNN models, which largely rely on the assumption of homophily, often struggle under heterophilous conditions due to the inherent low-pass filtering nature during message-passing among neighboring nodes. Motivated by these limitations, our research explores the dendritic structures found in real neurons, which dynamically amplify and inhibit signals. We demonstrate how the biophysical properties of dendrites enable GNNs to adaptively modulate the Dirichlet energy of graph representations based on the specific context, providing a theoretical justification for integrating neural dendrites into GNNs to improve learning on heterophilous graphs. In this work, we introduce the GDendrite framework, which dynamically adjusts GNN's output response to the various contexts. Our empirical studies, conducted across 13 homophilous and heterophilous benchmark datasets, show that our framework significantly enhances the performance of traditional GNNs in heterophilous settings and outperforms state-of-the-art models in most cases.
Understanding and reasoning about dynamics governed by physical laws through visual observation, akin to human capabilities in the real world, poses significant challenges. Currently, object-centric dynamic simulation methods, which emulate human behavior, have achieved notable progress but overlook two critical aspects: 1) the integration of physical knowledge into models. Humans gain physical insights by observing the world and apply this knowledge to accurately reason about various dynamic scenarios; 2) the validation of model adaptability across diverse scenarios. Real-world dynamics, especially those involving fluids and objects, demand models that not only capture object interactions but also simulate fluid flow characteristics. To address these gaps, we introduce SlotPi, a slot-based physics-informed object-centric reasoning model. SlotPi integrates a physical module based on Hamiltonian principles with a spatio-temporal prediction module for dynamic forecasting. Our experiments highlight the model's strengths in tasks such as prediction and Visual Question Answering (VQA) on benchmark and fluid datasets. Furthermore, we have created a real-world dataset encompassing object interactions, fluid dynamics, and fluid-object interactions, on which we validated our model's capabilities. The model's robust performance across all datasets underscores its strong adaptability, laying a foundation for developing more advanced world models.
Few-shot learning is crucial for tabular data, where the high cost of annotation often limits the availability of labeled samples. Despite its importance in real-world applications such as healthcare and finance, few-shot learning in tabular domains has received limited attention. To address this, we introduce range-limited augmentation, a novel augmentation strategy for contrastive learning that perturbs numerical features within predefined feature-specific ranges. Unlike conventional augmentations, which may result in false positive pairs during contrastive learning, our approach ensures semantic consistency by restricting augmentations to ranges. A quantitative analysis confirms that range-limited augmentation better preserves task-relevant information compared to existing augmentation techniques. Additionally, we propose FeSTa (Few-Shot Tabular classification benchmark), the first large-scale benchmark designed to systematically evaluate few-shot learning methods in tabular data. FeSTa includes 50 datasets and 32 algorithms spanning supervised, unsupervised, self-supervised, semi-supervised, and foundation models. Experiments on FeSTa show that range-limited augmentation consistently ranks among the top methods, achieving an average rank of 2.6 out of 32 in 1-shot classification, despite not relying on large-scale pretraining or complex architectures. The benchmark code is available in https://github.com/kyungeun-lee/festa.git.
Despite plentiful successes achieved by graph representation learning in various domains, the training of graph neural networks (GNNs) still remains tenaciously challenging due to the tremendous computational overhead needed for sizable graphs in practice. Recently, graph data distillation (GDD), which seeks to distill large graphs into compact and informative ones, has emerged as a promising technique to enable efficient GNN training. However, most existing GDD works rely on heuristics that align model gradients or representation distributions on condensed and original graphs, leading to compromised result quality, expensive training for distilling large graphs, or both. Motivated by this, this paper presents an efficient and effective GDD approach, ClustGDD. Under the hood, ClustGDD resorts to synthesizing the condensed graph and node attributes through fast and theoretically-grounded clustering that minimizes the within-cluster sum of squares and maximizes the homophily on the original graph. The fundamental idea is inspired by our empirical and theoretical findings unveiling the connection between clustering and empirical condensation quality using Fréchet Inception Distance, a well-known quality metric for synthetic images. Furthermore, to mitigate the adverse effects caused by the homophily-based clustering, ClustGDD refines the nodal attributes of the condensed graph with a small augmentation learned via class-aware graph sampling and consistency loss. Our extensive experiments exhibit that GNNs trained over condensed graphs output by ClustGDD consistently achieve superior or comparable performance to state-of-the-art GDD methods in terms of node classification on five benchmark datasets, while being orders of magnitude faster.
In supervised learning, decision trees are valued for their interpretability and performance. While greedy decision tree algorithms like CART remain widely used due to their computational efficiency, they often produce sub-optimal solutions with respect to a regularized training loss. Conversely, optimal decision tree methods can find better solutions but are computationally intensive and typically limited to shallow trees or binary features. We present Dynamic Programming Decision Trees (DPDT), a framework that bridges the gap between greedy and optimal approaches. DPDT relies on a Markov Decision Process formulation combined with heuristic split generation to construct near-optimal decision trees with significantly reduced computational complexity. Our approach dynamically limits the set of admissible splits at each node while directly optimizing the tree regularized training loss. Theoretical analysis demonstrates that DPDT can minimize regularized training losses at least as well as CART. Our empirical study shows on multiple datasets that DPDT achieves near-optimal loss with orders of magnitude fewer operations than existing optimal solvers. More importantly, extensive benchmarking suggests statistically significant improvements of DPDT over both CART and optimal decision trees in terms of generalization to unseen data. We demonstrate DPDT practicality through applications to boosting, where it consistently outperforms baselines. Our framework provides a promising direction for developing efficient, near-optimal decision tree algorithms that scale to practical applications.
Time series anomaly detection (TSAD) is a central topic in data mining, with many algorithms proposed, including deep learning and LLM-based methods. However, these are often evaluated on flawed benchmarks with trivial anomalies, unrealistic densities, mislabeled ground truth, and run-to-failure biases. Recent studies using higher-quality datasets show that simpler distance-based methods, such as discords, can outperform deep learning; However, they typically rely on sliding windows with fixed lengths, limiting flexibility. We introduce GDFlex (Generalized Discords with Flexible Subsequence Length), a discord-based method that detects anomalies across multiple time scales using length-normalized Euclidean distance. It also incorporates bias-correction mechanisms to address limitations in Euclidean distance and z-normalization. GDFlex achieves notable accuracy gains over existing methods while remaining scalable, interpretable, and efficient.
Learning with noisy labels (LNL) is essential for training deep neural networks with imperfect data. Meta-learning approaches have achieved success by using a clean unbiased labeled set to train a robust model. However, this approach heavily depends on the availability of a clean labeled meta-dataset, which is difficult to obtain in practice. In this work, we thus tackle the challenge of meta-learning for noisy label scenarios without relying on a clean labeled dataset. Our approach leverages the data itself while bypassing the need for labels. Building on the insight that clean samples effectively preserve the consistency of related data structures across the last hidden and the final layer, whereas noisy samples disrupt this consistency, we design the Cross-layer Information Divergence-based Meta Update Strategy (CLID-MU). CLID-MU leverages the alignment of data structures across these diverse feature spaces to evaluate model performance and use this alignment to guide training. Experiments on benchmark datasets with varying amounts of labels under both synthetic and real-world noise demonstrate that CLID-MU outperforms state-of-the-art methods. The code is released at https://github.com/ruofanhu/CLID-MU.
In this paper, we reveal that existing Differentially Private Graph Neural Networks (DP-GNNs) are not effective against Graph Reconstruction Attack (GRA). We further attribute the ineffectiveness of existing DP-GNNs against GRA to their unstructured perturbation mechanism, which only induces unidirectional shift in the embedding similarity distribution. Specifically, this perturbation mechanism tends to decrease the embedding similarity of all node pairs without significantly disrupting the relative ranking, thus allowing GRA to still reconstruct the original graph structure by leveraging the relative ranking of similarities. To address this, we propose a novel Differentially Private Graph Neural Network based on Structured Perturbation (GRASP). Specifically, we observe that independent noise tends to decrease the embedding similarity, while identical noise tends to increase it. By integrating these two types of noise using a Bernoulli technique, we introduce a simple yet effective structured perturbation mechanism, which promotes bidirectional shift in the embedding similarity distribution, thereby effectively disrupting the relative ranking and defending against GRA. Extensive experiments on eight benchmark datasets demonstrate that GRASP effectively defends against GRA. Furthermore, GRASP achieves a superior privacy-utility trade-off compared to existing graph structure protection methods. The implementation of GRASP is available at https://github.com/ZhiyuZone/GRASP/.
Revealing the underlying causal mechanisms in the real world is critical for scientific and technical progress. Despite advancements over the past decades, the lack of high-quality data and the inability of traditional causal discovery algorithms (TCDA) to fully comprehend the exact semantics of variables have long been major obstacles to the broader application of causal discovery. To address this issue, this paper proposes a novel causal modeling framework, LLM-CD, which integrates the metadata-based reasoning capabilities of large language models (LLMs) with the data-driven modeling abilities of TCDA for causal discovery. LLM-CD deeply couples the reasoning abilities of LLMs at various stages of TCDA, and enhances causal discovery through an iterative process. Due to the issues of overconfidence and hallucination in LLMs, LLM-CD quantifies and analyzes its uncertainty by incorporating evidence-based deep learning theory with the assumptions of TCDA. We utilize a large-scale de-identified real patient dataset provided by a hospital, a new dataset extracted from MIMIC-IV about the same disease (lung cancer), and two benchmark datasets to comprehensively evaluate LLM-CD. Extensive experimental results confirm the effectiveness and reliability of LLM-CD, with the highest improvement of 403.93% in the Recall and 25.77% in the Ratio metric across four datasets.
Zero-shot composed image retrieval (ZS-CIR) is a challenging task that aims to retrieve images similar to a composed query of a reference image and a description, without relying on training on triplet datasets. Existing methods for this task often rely on predefined, fixed retrieval processes that combine the image and the modified text through hand-crafted templates, which suffer from two main issues: non-adaptive retrieval queries and user-unfriendly retrieval processes. To address these limitations, we propose a novel framework - Automatic Multi-Agent Collaboration for Zero-Shot Composed Image Retrieval (AutoCIR). AutoCIR consists of three training-free agents - a planner, a retriever, and a corrector - that work together to iteratively identify and rectify mismatches. The planner guides the retriever by generating a customized target caption for the composed query and further refines this caption to resolve any semantic discrepancies based on feedback. The corrector, equipped with a chain-of-thought reasoning mechanism, conducts an in-depth evaluation of the retrieved results and generates appropriate self-correction actions. Extensive experiments on three benchmarks demonstrate that AutoCIR consistently outperforms previous competitive methods for ZS-CIR.
Recent advancements in time-series forecasting have highlighted the importance of frequency-domain modeling. However, deep learning models primarily operate in the time domain, limiting their ability to capture frequency-based patterns. Existing approaches normally introduce novel neural network architectures tailored to task-specific frequency properties, yet they often lack generalization and require extensive domain-specific adaptations. In this paper, we propose FAT, a novel pretraining framework that learns generalizable Frequency-Aware Time-series representations through self-supervised learning. The key idea of FAT is to pretrain any backbone model to directly extract generalizable frequency patterns from time-domain signals and encode them into robust representations-eliminating the need for architectural modifications or additional modules during inference. This is achieved via a frequency reformer that amplifies critical frequency components learned through self-supervision and enforces similarity between the original and frequency-reformed time-series representations produced by the encoder. In addition, recognizing that semantically equivalent time-series can exhibit different frequency expressions-analogous to how the same phrase is pronounced differently by different speakers-FAT introduces a Knowledge-Guided Frequency Reformer that unifies the expression of frequency patterns with the same underlying semantics and extends similarity constraints to frequency-invariant augmented samples to enhance robustness of learned representation. Experiments on 14 benchmark datasets across regression and classification tasks show that FAT consistently achieves state-of-the-art performance while maintaining robustness across diverse backbone models, significantly outperforming existing pretraining methods. Our code is available at https://github.com/JiaXiangfei/FAT.
Decoupling Spatio-Temporal Prediction: When Lightweight Large Models Meet Adaptive Hypergraphs
PDF ↗Spatio-temporal prediction is a pivotal task with broad applications in traffic management, climate monitoring, energy scheduling, etc. However, existing methodologies often struggle to balance model expressiveness and computational efficiency, especially when scaling to large real-world datasets. To tackle these challenges, we propose STH-SepNet (Spatio-Temporal Hypergraph Separation Networks), a novel framework that decouples temporal and spatial modeling to enhance both efficiency and precision. Therein, the temporal dimension is modeled using lightweight large language models, which effectively capture low-rank temporal dynamics. Concurrently, the spatial dimension is addressed through an adaptive hypergraph neural network, which dynamically constructs hyperedges to model intricate, higher-order interactions. A carefully designed gating mechanism is integrated to seamlessly fuse temporal and spatial representations. By leveraging the fundamental principles of low-rank temporal dynamics and spatial interactions, STH-SepNet offers a pragmatic and scalable solution for spatio-temporal prediction in real-world applications. Extensive experiments on large-scale real-world datasets across multiple benchmarks demonstrate the effectiveness of STH-SepNet in boosting predictive performance while maintaining computational efficiency. This work may provide a promising lightweight framework for spatio-temporal prediction, aiming to reduce computational demands and while enhancing predictive performance. Our code is avaliable at https://github.com/SEU-WENJIA/ST-SepNet-Lightweight-LLMs-Meet-Adaptive-Hypergraphs.
Spatio-temporal forecasting is crucial in various fields and requires a careful balance between identifying subtle patterns and filtering out noise. Vector quantization (VQ) appears well-suited for this purpose, as it quantizes input vectors into a set of codebook vectors or patterns. Although VQ has shown promise in various computer vision tasks, it surprisingly falls short in enhancing the accuracy of spatio-temporal forecasting. We attribute this to two main issues: inaccurate optimization due to non-differentiability and limited representation power caused by hard-VQ structure. To tackle these challenges, we introduce Differentiable Sparse Soft-Vector Quantization (SVQ), the first VQ method to enhance spatio-temporal forecasting. SVQ balances detail preservation with noise reduction, offering full differentiability and a solid foundation in sparse regression. The method employs a two-layer MLP and an extensive codebook to streamline the sparse regression process, significantly cutting computational costs while simplifying training and improving performance. Empirical studies on five spatio-temporal benchmark datasets show SVQ achieves state-of-the-art results, including a 7.9% improvement on the WeatherBench-S temperature dataset and an average mean absolute error reduction of 9.4% in video prediction benchmarks (Human3.6M, KTH, and KittiCaltech), along with a 17.3% enhancement in image quality (measured by LPIPS).