Retrieval-Augmented Generation (RAG) has been an effective approach to mitigate hallucinations in large language models (LLMs) by incorporating up-to-date and domain-specific knowledge. Recently, there has been a trend of storing up-to-date or copyrighted data in RAG knowledge databases instead of using it for LLM training. This practice has raised concerns about Membership Inference Attacks (MIAs), which aim to detect if a specific target document is stored in the RAG system's knowledge database so as to protect the rights of data producers. While research has focused on enhancing the trustworthiness of RAG systems, existing MIAs for RAG systems remain largely insufficient. Previous work either relies solely on the RAG system's judgment or is easily influenced by other documents or the LLM's internal knowledge, which is unreliable and lacks explainability. To address these limitations, we propose a Mask-Based Membership Inference Attacks (MBA) framework. Our framework first employs a masking algorithm that effectively masks a certain number of words in the target document. The masked text is then used to prompt the RAG system, and the RAG system is required to predict the mask values. If the target document appears in the knowledge database, the masked text will retrieve the complete target document as context, allowing for accurate mask prediction. Finally, we adopt a simple yet effective threshold-based method to infer the membership of target document by analyzing the accuracy of mask prediction. Our mask-based approach is more document-specific, making the RAG system's generation less susceptible to distractions from other documents or the LLM's internal knowledge. Extensive experiments demonstrate the effectiveness of our approach compared to existing baseline models.
论文检索
输入标题、作者或关键词，从 3,655 篇学术成果中精准定位
Rumor Detection on Social Media with Reinforcement Learning-based Key Propagation Graph Generator
PDF ↗The spread of rumors on social media, particularly during significant events like the US elections and the COVID-19 pandemic, poses a serious threat to social stability and public health. Current rumor detection methods primarily rely on propagation graphs to improve the model performance. However, the effectiveness of these methods is often compromised by noisy and irrelevant structures in the propagation process. To tackle this issue, techniques such as weight adjustment and data augmentation have been proposed. However, they depend heavily on rich original propagation structures, limiting their effectiveness in handling rumors that lack sufficient propagation information, especially in the early stages of dissemination. In this work, we introduce the Key Propagation Graph Generator (KPG), a novel reinforcement learning-based framework, that generates contextually coherent and informative propagation patterns for events with insufficient topology information and identifies significant substructures in events with redundant and noisy propagation structures. KPG comprises two key components: the Candidate Response Generator (CRG) and the Ending Node Selector (ENS). CRG learns latent variable distributions from refined propagation patterns to eliminate noise and generate new candidates for ENS, while ENS identifies the most influential substructures in propagation graphs and provides training data for CRG. Furthermore, we develop an end-to-end framework that utilizes rewards derived from a pre-trained graph neural network to guide the training process. The resulting key propagation graphs are then employed in downstream rumor detection tasks. Extensive experiments conducted on four datasets demonstrate that KPG outperforms current state-of-the-art methods.
ColaCare: Enhancing Electronic Health Record Modeling through Large Language Model-Driven Multi-Agent Collaboration
PDF ↗We introduce ColaCare, a framework that enhances Electronic Health Record (EHR) modeling through multi-agent collaboration driven by Large Language Models (LLMs). Our approach seamlessly integrates domain-specific expert models with LLMs to bridge the gap between structured EHR data and text-based reasoning. Inspired by the Multidisciplinary Team (MDT) approach used in clinical settings, ColaCare employs two types of agents: DoctorAgents and a MetaAgent, which collaboratively analyze patient data. Expert models process and generate predictions from numerical EHR data, while LLM agents produce reasoning references and decision-making reports within the MDT-driven collaborative consultation framework. The MetaAgent orchestrates the discussion, facilitating consultations and evidence-based debates among DoctorAgents, simulating diverse expertise in clinical decision-making. We additionally incorporate the Merck Manual of Diagnosis and Therapy (MSD) medical guideline within a retrieval-augmented generation (RAG) module for medical evidence support, addressing the challenge of knowledge currency. Extensive experiments conducted on three EHR datasets demonstrate ColaCare's superior performance in clinical mortality outcome and readmission prediction tasks, underscoring its potential to revolutionize clinical decision support systems and advance personalized precision medicine. All code, case studies and a questionnaire are available at the project website: https://colacare.netlify.app.
Augmenting large language models (LLMs) with external tools has emerged as a promising approach to extend their utility, enabling them to solve practical tasks. Previous methods manually parse tool documentation and create in-context demonstrations, transforming tools into structured formats for LLMs to use in their step-by-step reasoning. However, this manual process requires domain expertise and struggles to scale to large toolsets. Additionally, these methods rely heavily on ad-hoc inference techniques or special tokens to integrate free-form LLM generation with tool-calling actions, limiting the LLM's flexibility in handling diverse tool specifications and integrating multiple tools. In this work, we propose AutoTools, a framework that enables LLMs to automate the tool-use workflow. Specifically, the LLM automatically transforms tool documentation into callable functions, verifying syntax and runtime correctness. Then, the LLM integrates these functions into executable programs to solve practical tasks, flexibly grounding tool-use actions into its reasoning processes. Extensive experiments on existing and newly collected, more challenging benchmarks illustrate the superiority of our framework. Inspired by these promising results, we further investigate how to improve the expertise of LLMs, especially open-source LLMs with fewer parameters, within AutoTools. Thus, we propose the AutoTools-Learning approach, training the LLMs with three learning tasks on 34k instances of high-quality synthetic data, including documentation understanding, relevance learning, and function programming. Fine-grained results validate the effectiveness of our overall training approach and each individual task. Our methods are an important step towards the use of LLMs for solving real-world tasks with external tools.
The core of the general recommender systems lies in learning high-quality embedding representations of users and items to investigate their positional relations in the feature space. Unfortunately, data sparsity caused by difficult-to-access interaction data severely limits the effectiveness of recommender systems. Faced with such a dilemma, various types of self-supervised learning methods have been introduced into recommender systems in an attempt to alleviate the data sparsity through distribution modeling or data augmentation. However, most data augmentation relies on elaborate manual design, which is not only not universal, but the bloated and redundant augmentation process may significantly slow down model training progress. To tackle these limitations, we propose a novel Dual Mixing-based Recommendation Framework (MixRec) to empower data augmentation as we wish. Specifically, we propose individual mixing and collective mixing, respectively. The former aims to provide a new positive sample that is unique to the target (user or item) and to make the pair-wise recommendation loss benefit from it, while the latter aims to portray a new sample that contains group properties in a batch. The two mentioned mixing mechanisms allow for data augmentation with only one parameter that does not need to be set multiple times and can be done in linear time complexity. Besides, we propose the dual-mixing contrastive learning to maximize the utilization of these new-constructed samples to enhance the consistency between pairs of positive samples. Experimental results on four real-world datasets demonstrate the advantages of MixRec in terms of effectiveness, simplicity, efficiency, and scalability.
Graph anomaly detection has gained significant attention across various domains, particularly in critical applications like fraud detection in e-commerce platforms and insider threat detection in cybersecurity. Usually, these data are composed of multiple types (e.g., user information and transaction records for financial data), thus exhibiting view heterogeneity. However, in the era of big data, the heterogeneity of views and the lack of label information pose substantial challenges to traditional approaches. Existing unsupervised graph anomaly detection methods often struggle with high-dimensionality issues, rely on strong assumptions about graph structures or fail to handle complex multi-view graphs. To address these challenges, we propose a cluster aware multi-view graph anomaly detection method, called CARE. Our approach captures both local and global node affinities by augmenting the graph's adjacency matrix with the pseudo-label (i.e., soft membership assignments) without any strong assumption about the graph. To mitigate potential biases from the pseudo-label, we introduce a similarity-guided loss. Theoretically, we show that the proposed similarity-guided loss is a variant of contrastive learning loss, and we present how this loss alleviates the bias introduced by pseudo-label with the connection to graph spectral clustering. Experimental results on several datasets demonstrate the effectiveness and efficiency of our proposed framework. Specifically, CARE outperforms the second-best competitors by more than 39% on the Amazon dataset with respect to AUPRC and 18.7% on the YelpChi dataset with respect to AUROC. The code of our method is available at the GitHub link: https://github.com/zhenglecheng/CARE-demo.
Heterophilic Graph Neural Networks (HGNNs) have shown promising results for semi-supervised learning tasks on graphs. Notably, most real-world heterophilic graphs are composed of a mixture of nodes with different neighbor patterns, exhibiting local node-level homophilic and heterophilic structures. However, existing works are only devoted to designing better unified HGNN backbones for node classification tasks on heterophilic and homophilic graphs simultaneously, and their analyses of HGNN performance concerning nodes are only based on the determined data distribution without exploring the effect caused by the difference of structural pattern between training and testing nodes. How to learn invariant node representations on heterophilic graphs to handle this structure difference or distribution shifts remains unexplored. In this paper, we first discuss the limitations of previous graph-based invariant learning methods in addressing the heterophilic graph structure distribution shifts from the perspective of data augmentation. Then, we propose HEI, a framework capable of generating invariant node representations through incorporating Heterophily information, the node's estimated neighbor pattern, to infer latent Environments without augmentation, which are then used for Invariant prediction. We provide detailed theoretical guarantees to clarify the reasonability of HEI. Extensive experiments on various benchmarks and backbones can also demonstrate the effectiveness and robustness of our method compared with existing state-of-the-art baselines.
Beyond Single Tabs: A Transformative Few-Shot Approach to Multi-Tab Website Fingerprinting Attacks
PDF ↗Website Fingerprinting (WF) attacks allow passive eavesdroppers to deduce the websites a user visits by analyzing encrypted traffic, threatening user privacy. While current WF attacks achieve high accuracy, they typically assume single-tab browsing, which is unrealistic as users often open multiple tabs, creating mixed traffic. Existing multi-tab WF approaches require large datasets and frequent retraining due to evolving website content, limiting their practicality. In this paper, we introduce Few-shot Multi-tab Website Fingerprinting (FMWF), a novel approach designed to address the limitations of existing multi-tab WF attacks. FMWF directly tackles the challenges of mixed, overlapping traffic traces generated from multi-tab browsing, leveraging two key innovations: (1) an advanced data augmentation technique that synthesizes realistic multi-tab traffic sequences from easily collected single-tab traces, thereby dramatically reducing the need for large-scale real-world traffic data; and (2) a powerful fine-tuning algorithm based on transfer learning that adapts pre-trained models to new, multi-tab environments with minimal additional data. This two-stage framework enables FMWF to capture the complex effectively, overlapping traffic patterns inherent in multi-tab browsing while maintaining a high level of flexibility and significantly lowering computational and data collection burdens. Our experiments, conducted using real traffic traces collected from three widely-used browsers-Microsoft Edge, Google Chrome, and Tor Browser-highlight the superior performance of FMWF in both closed-world and open-world scenarios. Notably, FMWF achieves a minimum 12.3% improvement in accuracy compared to ARES (SP'23) [7], TMWF (CCS'23) [13], and BAPM (ACSAC'21) [10] in the open-world scenario. The code with related datasets is available at https://github.com/WW-Meng/FMWF.
Graph-based social recommender systems utilize user-item interaction graphs and user-user social graphs to model user preferences. However, their performance can be limited by redundant and noisy information in these two graphs. Although several recommender studies on data denoising exist, most either rely on heuristic assumptions, which limit their adaptability, or use a single model that combines denoising and recommendation, potentially imposing substantial demands on the model capacity. To address these issues, we propose a dual Graph Denoising Social Recommender (GDSR), which consists of two steps: graph denoising and user preference prediction. First, we design a denoising module which exploits a dual diffusion model to alleviate noises in the interaction and social graphs by performing multi-step noise diffusion and removal. We develop three kinds of conditions to guide our dual graph diffusion paradigm and propose a cross-domain signal guidance mechanism to enhance the structure of denoised graphs. Second, we devise a recommender module that employs a dual graph learning structure on denoised graphs to generate recommendations. Moreover, we use additional supervision signals from the diffusion-enhanced data augmentation to introduce a graph contrastive learning task, enhancing the recommender module's representation quality and robustness. Experiment results show the effectiveness of our GDSR.
Explainable recommendation has demonstrated significant advantages in informing users about the logic behind recommendations, thereby increasing system transparency, effectiveness, and trustworthiness. To provide personalized and interpretable explanations, existing works often combine the generation capabilities of large language models (LLMs) with collaborative filtering (CF) information. CF information extracted from the user-item interaction graph captures the user behaviors and preferences, which is crucial for providing informative explanations. However, due to the complexity of graph structure, effectively extracting the CF information from graphs still remains a challenge. Moreover, existing methods often struggle with the integration of extracted CF information with LLMs due to its implicit representation and the modality gap between graph structures and natural language explanations. To address these challenges, we propose G-Refer, a framework using Graph Retrieval-augmented large language models (LLMs) for explainable recommendation. Specifically, we first employ a hybrid graph retrieval mechanism to retrieve explicit CF signals from both structural and semantic perspectives. The retrieved CF information is explicitly formulated as human-understandable text by the proposed graph translation and accounts for the explanations generated by LLMs. To bridge the modality gap, we introduce knowledge pruning and retrieval-augmented fine-tuning to enhance the ability of LLMs to process and utilize the retrieved CF information to generate explanations. Extensive experiments show that G-Refer achieves superior performance compared with existing methods in both explainability and stability. Codes and data are available at https://github.com/Yuhan1i/G-Refer.
Machine learning traditionally emphasizes developing models for given datasets, but real-world data is often messy, making model improvement insufficient for enhancing performance. AI for data editing (AI4DE) is an emerging field that systematically improves datasets, leading to significant practical ML advancements. While experienced data scientists have manually refined datasets through trial-and-error and intuition, AI4DE approaches data enhancement as a systematic engineering discipline. AI4DE represents a shift from focusing on models to the underlying data used for training and evaluation. Despite the dominance of common model architectures and predictable scaling rules, building and using datasets remain labor-intensive and costly, lacking infrastructure and best practices. The AI4DE movement aims to develop efficient, high-productivity open data engineering tools for modern ML systems. This workshop seeks to foster an interdisciplinary AI4DE community to address practical data challenges, including data collection, generation, labeling, preprocessing, augmentation, quality evaluation, debt, and governance. By defining and shaping the AI4DE movement, this workshop aims to influence the future of AI and ML, inviting interested parties to contribute through paper submissions
Inspired by the success of large language models (LLMs), there is a significant research shift from traditional graph learning methods to LLM-based graph frameworks, formally known as GraphLLMs. GraphLLMs leverage the reasoning power of LLMs by integrating three key components: the textual attributes of input nodes, the structural information of node neighborhoods, and task-specific prompts that guide decision-making. Despite their promise, the robustness of GraphLLMs against adversarial perturbations remains largely unexplored-a critical concern for deploying these models in high-stakes scenarios. To bridge the gap, we introduce TrustGLM, a comprehensive study evaluating the vulnerability of GraphLLMs to adversarial attacks across three dimensions: text, graph structure, and prompt manipulations. We implement state-of-the-art attack algorithms from each perspective to rigorously assess model resilience. Through extensive experiments on six benchmark datasets from diverse domains, our findings reveal that GraphLLMs are highly susceptible to text attacks that merely replace a few semantically similar words in a node's textual attribute. We also find that standard graph structure attack methods can significantly degrade model performance, while random shuffling of the candidate label set in prompt templates leads to substantial performance drops. Beyond characterizing these vulnerabilities, we investigate defense techniques tailored to each attack vector through data-augmented training and adversarial training, which show promising potential to enhance the robustness of GraphLLMs. We hope that our open-sourced library will facilitate rapid, equitable evaluation and inspire further innovative research in this field. The benchmark code can be found in https://github.com/Palasonic5/TrustGLM.git.
Capillary Dataset: A Dataset of Nail-fold Capillaries Captured by Microscopy for Diabetes Detection
PDF ↗Diabetes mellitus is a chronic condition marked by insufficient insulin utilization or production, causing metabolic dysregulation. If not controlled, it can cause serious complications that affect major organ systems such as the cardiovascular and ocular systems. Early diagnosis is essential for timely interventions to ensure effective glycemic control and lower the risk of complications. The present study introduced a robust and comprehensive dataset derived from nail-fold capillaroscopy. This dataset, which includes imaging and some video data from 126 individuals diagnosed with type 2 diabetes mellitus (T2DM) alongside 76 healthy controls, consisted of 3283 images obtained from diabetic subjects and 3412 images from non-diabetic participants. All images were acquired at 390× high magnification with a high resolution of 640×480 pixels, including corresponding video data, ensuring a thorough and detailed dataset for our study. The dataset was organized into two analytical tracks. The first track focused on nail-fold morphology, classifying images into four types: hairpin, crossing, tortuous, and bushy. After filtering out duplicates and low-quality images, 1279 images were selected for analysis. The second track involved binary classification for the detection of diabetes, differentiating healthy individuals from people with diabetes. This utilized the complete dataset and the augmented versions created by combining multiple images into composite formats to enhance feature representation. Additionally, we evaluated several benchmark deep-learning models such as Vision Transformers (ViT) and Convolutional Neural Networks (CNNs) for morphological classification and diabetes detection tasks. This analysis illuminated current model performance, highlighted challenges, and paved the way for future research opportunities. Importantly, this dataset is critical and holds significant potential for advancing non-invasive automated diagnostic methodologies in diabetes-related nail-fold capillary research, offering a promising future for diabetes research. The dataset is available at: https://huggingface.co/datasets/HanaNguyen/Capillary-Dataset. The github: https://github.com/urgonguyen/Capillarydataset.git.
WikiRAG: Revisiting Wikidata KGC Datasets with Community Updates and Retrieval-Augmented Generation
PDF ↗Link prediction is an important task for knowledge graph completion and curation, and it has received significant attention from the research community. However, researchers often train and evaluate new models on small or outdated datasets that do not reflect the current state of knowledge, thereby disregarding new information and the rich textual content often linked to knowledge graphs. As a result, many opportunities to leverage these dimensions are missed. We introduce WikiRAG, a framework for knowledge completion and evaluation derived from Wikidata and Wikipedia, which enables research integrating retrieval techniques and large language models. Our framework combines the following contributions: (i) We revisit the Wikidata5M dataset by updating it to reflect the current state of Wikidata and providing automated tools for its periodic maintenance. (ii) We enrich the dataset with long-form textual content sourced from Wikipedia, enabling research that goes beyond traditional graph structures and shallow text methods toward dense retrieval techniques. (iii) We propose a simple yet effective baseline that leverages retrieval-augmented generation, demonstrating the utility of the dataset and integrating language model capabilities for link prediction. The revised dataset, coined Wikidata5M-RE, shows that the original graph grew by roughly 50% in the number of edges, while 10% of the edges have been removed. A comparative analysis of classic methods demonstrates that these changes can impact downstream task evaluation. Finally, our evaluation of WikiRAG's KGC method shows an improvement of up to 9% in link prediction accuracy over state-of-the-art baselines, setting the stage for a new avenue in knowledge completion that uses deep information extraction. The source code, data, and other artifacts have been made available on the project website: https://github.com/colab-nyuad/WikiRAG
An emerging type of fraud involves malicious senders exploiting the blind shipment and cash-on-delivery (COD) mechanisms by dispatching large volumes of unsolicited, low-cost parcels. If unsuspecting receivers accept these parcels, they pay for both shipping and goods; otherwise, logistics providers bear the round-trip shipping costs. Existing detection techniques, which rely on extensive labeled cases, struggle with this emerging fraud because receivers' unawareness and low transaction values discourage complaints, resulting in few confirmed cases. Therefore, we propose leveraging receivers' complaints, though not initially collected for fraud detection, to uncover subtle indicators of fraud patterns, while addressing three challenges: (C1) noise-rich dialogues(C2) data privacy concerns, and (C3) ever-evolving fraud patterns. To address them, we design BLOFF, a Blind shipment detection Framework for LO gistics Fraud powered by large language models (LLMs). Specifically, BLOFF includes three components: i) Sensitivity Anonymization to protect sensitive user information; ii) Dialogue Profile Distillation to transform informal dialogues into structured representation, addressing C1, and distill knowledge from a teacher LLM (GPT-4o) to a lightweight student LLM (ChatGLM4-9B), addressing C2; ii) Multi-faceted Context Augmentation to enhance the interpretation of fraud signatures and adaptation of evolving patterns, addressing C3. We evaluate BLOFF on about 56,000 complaints records collected from JD Logistics between January and November 2024. Results show that BLOFF outperforms state-of-the-art methods, achieving a 10.19% improvement in precision. Furthermore, during its real-world deployment in December 2024, BLOFF identified over 90 fraudulent parcels with a 91.4% precision.
Complicated Semantic Alignment for Long-Tail Query Rewriting in Taobao Search Based on Large Language Model
PDF ↗In the realm of e-commerce search, semantic matching has consistently been a core issue, as it directly affects user experience and company revenue. However, users' queries often fail to effectively retrieve relevant products due to discrepancies between the user's expression habits and product names written by merchants. Even existing large language model (LLM) based query rewriting methods can bridge the semantic gap for most queries, they are still ineffective for long-tail queries with complicated semantic. In this paper, we propose Complicated Semantic Alignment Query Rewrite(CSA-QR) framework, which mitigates the semantic differences in long-tail queries with complicated semantics. CSA-QR comprises three stages: high-quality supervised fine-tuning (SFT) dataset generation, multi-dimensional alignment dataset generation, and binary feedback Proximal Policy Optimization (PPO) for reinforcement alignment. Initially, we utilize general large language models to generate rewrite candidates, followed by manual annotation to discriminate the candidates, then use the retrieval augmentation generation (RAG) based on existing annotations to produce a higher quality SFT dataset. Subsequently, we decouple the feedback data into user semantic consistency and merchant expression consistency dimensions to collect multi-dimensional alignment data. Finally, we introduce a binary feedback method to train the reward model, enabling it to better guide alignment training within our context. We also identify a set of more appropriate reward model evaluation metrics to guide our iterations. Offline experiments demonstrate the effectiveness of this method in improving retrieval performance. Online A/B tests reveal that our method significantly boosts critical metrics such as product click-through rate (CTR), gross merchandise volume (GMV) and number of transaction (#Trans) for long-tail complicated queries. CSA-QR has been deployed on Taobao, one of China's most popular online shopping platforms, since September 2024.
Satellite images play a crucial role in weather analytics, and recent advancements in satellite technology have significantly enhanced the accuracy and reliability of weather predictions. In this paper, we introduce SkySearch, a large-scale satellite video search system deployed at the Korean Meteorological Administration (KMA). SkySearch is designed to aid weather experts in making timely and accurate forecasts by rapidly and precisely searching for satellite videos in the database that resemble current weather conditions. SkySearch employs self-supervised learning to compress large volumes of high-resolution satellite videos into low-dimensional embeddings, addressing the absence of labeled satellite data. Within this latent space, relationships between videos are modeled as a graph, enabling efficient searches. Given a query video, SkySearch rapidly identifies a small subset of similar videos by traversing the graph. When the query video represents the current conditions, it can optionally be augmented with predicted future frames to search for videos that reflect both the current conditions and expected evolution. Finally, the ranked list of videos is provided to weather forecasters through a user-friendly interface. We demonstrate that deployment and empirical effectiveness of SkySearch through both numerical and qualitative evaluations. In summary, SkySearch is: (a) Scalable: processes queries from a large-scale database of satellite images spanning over a decade and delivers results within seconds(b) Accurate: returns numerically and qualitatively similar videos to the query video, and (c) Label-free: does not require labeled videos.
In federated learning (FL), the assumption that datasets from different devices are independent and identically distributed (i.i.d.) often does not hold due to user differences, and the presence of various data modalities across clients makes using a single model impractical. Personalizing certain parts of the model can effectively address these issues by allowing those parts to differ across clients, while the remaining parts serve as a shared model. However, we found that partial model personalization may exacerbate client drift (each client's local model diverges from the shared model), thereby reducing the effectiveness and efficiency of FL algorithms. We propose an FL framework based on the alternating direction method of multipliers (ADMM), referred to as FedAPM, to mitigate client drift. We construct the augmented Lagrangian function by incorporating first-order and second-order proximal terms into the objective, with the second-order term providing fixed correction and the first-order term offering compensatory correction between the local and shared models. Our analysis demonstrates that FedAPM, by using explicit estimates of the Lagrange multiplier, is more stable and efficient in terms of convergence compared to other FL frameworks. We establish the global convergence of FedAPM training from arbitrary initial points to a stationary point, achieving three types of rates: constant, linear, and sublinear, under mild assumptions. We conduct experiments using four heterogeneous and multimodal datasets with different metrics to validate the performance of FedAPM. Specifically, FedAPM achieves faster and more accurate convergence, outperforming the SOTA methods with average improvements of 12.3% in test accuracy, 16.4% in F1 score, and 18.0% in AUC while requiring fewer communication rounds.
With the growing availability of large data repositories within and across organizations, it is becoming feasible to selectively acquire data in the wild for data augmentation, tailored to specific downstream tasks. However, current methodologies concentrate primarily on single-dimensional augmentation tasks for tabular data, such as increasing the number of data points or enriching features. These approaches are not designed for trajectory data, which cannot fully utilize its spatio-temporal characteristics, resulting in suboptimal performance. To address this gap, we present Toast, an iterative framework of task-oriented multi-dimensional augmentation for spatio-temporal trajectory data. Toast supports all three augmentation operations and consists of three core components: (1) augmentation candidate search, which identifies unionable or joinable augmentation candidates in the wild; (2) augmentation candidate selection, which defines and selects the ''best'' augmentations that offer the greatest benefit to downstream tasks; (3) augmentation type selection, which determines the most cost-effective type of augmentation to perform based on a cost model. Extensive experiments on four classical trajectory data mining applications and datasets demonstrate that Toast can enhance model performance by 5%-15% and achieve more than a 2× improvement in efficiency.
Offline reinforcement learning (RL) aims to learn policies without online explorations. To enlarge the training data, model-based offline RL learns a dynamics model which is utilized as a virtual environment to generate simulation data and enhance policy learning. However, existing data augmentation methods for offline RL suffer from (i) trivial improvement from short-horizon simulation; and (ii) the lack of evaluation and correction for generated data, leading to low-qualified augmentation. In this paper, we propose offline trajectory optimization for offline reinforcement learning (OTTO). The key motivation is to conduct long-horizon simulation and then utilize model uncertainty to evaluate and correct the augmented data. Specifically, we propose an ensemble of Transformers, a.k.a. World Transformers, to predict environment state dynamics and the reward function. Three strategies are proposed to use World Transformers to generate long-horizon trajectory simulation by perturbing the actions in the offline data. Then, an uncertainty-based World Evaluator is introduced to firstly evaluate the confidence of the generated trajectories and then perform the correction for low-confidence data. Finally, we jointly use the original data with the corrected augmentation data to train an offline RL algorithm. OTTO serves as a plug-in module and can be integrated with existing model-free offline RL methods. Experiments on various benchmarks show that OTTO can effectively improve the performance of representative offline RL algorithms, including in complex environments with sparse rewards like AntMaze. Codes are available at https://github.com/ZiqiZhao1/OTTO