We study the problem of selecting the best m units from a set of n as m / n → α ∈ (0, 1), where noisy, heteroskedastic measurements of the units' true values are available and the decision-maker wishes to maximize the aggregate true value of the units selected. Given a parametric prior distribution, the empirical Bayes decision rule incurs O p(n^-1) regret relative to the Bayesian oracle that knows the true prior. More generally, if the error in the estimated prior is of order O p(rn), regret is Op(rn2). In this sense selection of the best units is fundamentally easier than estimation of their values. We show this regret bound is sharp in the parametric case, by giving an example in which it is attained. Using priors calibrated from a dataset of over four thousand internet experiments, we confirm that empirical Bayes methods perform well in detecting the best treatments with only a modest number of experiments.
论文检索
输入标题、作者或关键词，从 844 篇学术成果中精准定位
Online advertising has become one of the most successful business models of the internet era. Impression opportunities are typically allocated through real-time auctions, where advertisers bid to secure advertisement slots. Deciding the best bid for an impression opportunity is challenging, due to the stochastic nature of user behavior and the variability of advertisement traffic over time. In this work, we propose a framework for training auto-bidding agents in multi-slot second-price auctions to maximize acquisitions (e.g., clicks, conversions) while adhering to budget and cost-per-acquisition (CPA) constraints. We exploit the insight that, after an advertisement campaign concludes, determining the optimal bids for each impression opportunity can be framed as a multiple-choice knapsack problem (MCKP) with a nonlinear objective. We propose an ''oracle'' algorithm that identifies a near-optimal combination of impression opportunities and advertisement slots, considering both past and future advertisement traffic data. This oracle solution serves as a training target for a student network which bids having access only to real-time information, a method we term Oracle Imitation Learning (OIL). Through numerical experiments, we demonstrate that OIL achieves superior performance compared to both online and offline reinforcement learning algorithms, offering improved sample efficiency. Notably, OIL shifts the complexity of training auto-bidding agents from crafting sophisticated learning algorithms to solving a nonlinear constrained optimization problem efficiently. Code available at https://github.com/sony/oil
Zero-shot composed image retrieval (ZS-CIR) is a challenging task that aims to retrieve images similar to a composed query of a reference image and a description, without relying on training on triplet datasets. Existing methods for this task often rely on predefined, fixed retrieval processes that combine the image and the modified text through hand-crafted templates, which suffer from two main issues: non-adaptive retrieval queries and user-unfriendly retrieval processes. To address these limitations, we propose a novel framework - Automatic Multi-Agent Collaboration for Zero-Shot Composed Image Retrieval (AutoCIR). AutoCIR consists of three training-free agents - a planner, a retriever, and a corrector - that work together to iteratively identify and rectify mismatches. The planner guides the retriever by generating a customized target caption for the composed query and further refines this caption to resolve any semantic discrepancies based on feedback. The corrector, equipped with a chain-of-thought reasoning mechanism, conducts an in-depth evaluation of the retrieved results and generates appropriate self-correction actions. Extensive experiments on three benchmarks demonstrate that AutoCIR consistently outperforms previous competitive methods for ZS-CIR.
Graph Neural Networks (GNNs) have gained traction in Graph-based Machine Learning as a Service (GMLaaS) platforms, yet they remain vulnerable to graph-based model extraction attacks (MEAs), where adversaries reconstruct surrogate models by querying the victim model. Existing defense mechanisms, such as watermarking and fingerprinting, suffer from poor real-time performance, susceptibility to evasion, or reliance on post-attack verification, making them inadequate for handling the dynamic characteristics of graph-based MEA variants. To address these limitations, we propose ATOM, a novel real-time MEA detection framework tailored for GNNs. ATOM integrates sequential modeling and reinforcement learning to dynamically detect evolving attack patterns, while leveraging k-core embedding to capture the structural properties, enhancing detection precision. Furthermore, we provide theoretical analysis to characterize query behaviors and optimize detection strategies. Extensive experiments on multiple real-world datasets demonstrate that ATOM outperforms existing approaches in detection performance, maintaining stable across different time steps, thereby offering a more effective defense mechanism for GMLaaS environments. Our source code is available at https://github.com/LabRAI/ATOM.
Recent advancements in time-series forecasting have highlighted the importance of frequency-domain modeling. However, deep learning models primarily operate in the time domain, limiting their ability to capture frequency-based patterns. Existing approaches normally introduce novel neural network architectures tailored to task-specific frequency properties, yet they often lack generalization and require extensive domain-specific adaptations. In this paper, we propose FAT, a novel pretraining framework that learns generalizable Frequency-Aware Time-series representations through self-supervised learning. The key idea of FAT is to pretrain any backbone model to directly extract generalizable frequency patterns from time-domain signals and encode them into robust representations-eliminating the need for architectural modifications or additional modules during inference. This is achieved via a frequency reformer that amplifies critical frequency components learned through self-supervision and enforces similarity between the original and frequency-reformed time-series representations produced by the encoder. In addition, recognizing that semantically equivalent time-series can exhibit different frequency expressions-analogous to how the same phrase is pronounced differently by different speakers-FAT introduces a Knowledge-Guided Frequency Reformer that unifies the expression of frequency patterns with the same underlying semantics and extends similarity constraints to frequency-invariant augmented samples to enhance robustness of learned representation. Experiments on 14 benchmark datasets across regression and classification tasks show that FAT consistently achieves state-of-the-art performance while maintaining robustness across diverse backbone models, significantly outperforming existing pretraining methods. Our code is available at https://github.com/JiaXiangfei/FAT.
The combination of temporal graph neural networks (TGNNs) and relative positional features is effective for dynamic link prediction tasks because it improves the modeling of relationships between nodes. We summarize the positional feature as Temporal Relative Positional Encoding (TRPE), which incorporates local subgraph computation and temporal decay effects compared to RPE in static graph learning. However, existing dynamic graph learning methods with TRPE are limited by high computational costs and poor scalability due to information loss in neighborhood compression. To address this, we introduce a scalable framework that enhances TRPE computation by integrating it with temporal clusters. Our method replaces high-order relative position information with cluster-based computation to reduce computation costs. Each node is assigned to a cluster based on its structural role, rather than just randomly sketched, reducing information loss during neighborhood compression. TGNNs with our proposed framework are more expressive than existing dynamic graph learning methods and offer greater scalability than existing TRPE models. Our experimental results on seven standard temporal link prediction benchmarks demonstrate that our proposed model achieves comparable or superior performance compared to state-of-the-art models.
Graph Neural Networks (GNNs) are widely used in various applications and consist of two main operations: Propagation and Transformation. By permuting and combining these two operations, we can obtain different GNN architectures for different downstream tasks. Despite the effectiveness and popularity of GNNs, their training is time-consuming and memory-intensive, which limits their practical usage. Various methods have been proposed to tackle this issue, such as sampling and distributed training. In this work, we carefully revisit the two basic operations of GNNs and observe two types of redundancy in these operations: 1) propagation data redundancy, and 2) transformation computation redundancy. To address these problems, we propose CompressGNN, a framework that leverages hierarchical compression to accelerate GNNs by reducing data and computational redundancy. The framework comprises a propagation compression module and a transformation compression module. In the propagation compression module, we compress the graph structure to eliminate data redundancy. In the transformation compression module, we achieve computation compression by compressing similar node features at runtime. Extensive experiments demonstrate that our method achieves an average speedup of 2.32× across multiple datasets and models.
Time series anomaly detection has garnered significant research attention due to growing demands for temporal data monitoring across diverse domains. Despite the rapid advent of unsupervised anomaly detection models, existing approaches face two critical challenges in understanding the mechanisms of reconstruction-based models when handling diverse temporal dependencies: (1) the insufficient exploration of complex inter-timestamp relationships encompassing both short-term and long-term dependencies, and (2) the lack of integrated frameworks for jointly learning short-term patterns and long-term temporal characteristics. To address these challenges, we propose the novel Multi-Scale Hypergraph Transformer (MSHTrans), which leverages the capacity of hypergraphs for modeling multi-order temporal dependencies. Particularly, our method employs multi-scale downsampling to derive complementary fine-grained and coarse-grained representations, integrated with trainable hypergraph neural networks that can adaptively learn inter-timestamp relationships. The framework further integrates time series decomposition to systematically extract periodic and trend components from multi-granular features, thereby enhancing long-term dependency modeling. Through synergistic integration of learned short-term patterns and long-term temporal structures, the model achieves comprehensive time series reconstruction for effective anomaly detection. Extensive experiments demonstrate that MSHTrans outperforms state-of-the-art competitors with an average performance improvement of 8.21% (without point adjustment) and 3.52% (with point adjustment).
The widespread adoption of location-based services has led to the generation of vast amounts of mobility data, providing significant opportunities to model user movement dynamics within urban environments. Recent advancements have focused on adapting Large Language Models (LLMs) for mobility analytics. However, existing methods face two primary limitations: inadequate semantic representation of locations (i.e., discrete IDs) and insufficient modeling of mobility signals within LLMs (i.e., single templated instruction fine-tuning). To address these issues, we propose QT-Mob, a novel framework that significantly enhances LLMs for mobility analytics. QT-Mob introduces a location tokenization module that learns compact, semantically rich tokens to represent locations, preserving contextual information while ensuring compatibility with LLMs. Furthermore, QT-Mob incorporates a series of complementary fine-tuning objectives that align the learned tokens with the internal representations in LLMs, improving the model's comprehension of sequential movement patterns and location semantics. The proposed QT-Mob framework not only enhances LLMs' ability to interpret mobility data but also provides a more generalizable approach for various mobility analytics tasks. Experiments on three real-world dataset demonstrate the superior performance in both next-location prediction and mobility recovery tasks, outperforming existing deep learning and LLM-based methods.
Enhancer: A Distribution-Aware Framework with Temporal-Relational Meta-Learning for Stock Prediction
Accurate stock prediction is critical for portfolio management, where learning to adapt to market changes is the key to sustainable profitability. Financial markets, as complex interactive systems, exhibit evolution in both temporal dynamics and relational structures. While current temporal-relational models have achieved remarkable success in stock prediction, they face fundamental challenges in learning and adapting to market changes, particularly the systematic shifts in temporal and relational distributions that challenge the i.i.d. assumption underlying model training. In this study, we pioneer the research of temporal-relational distribution shifts in stock prediction and introduce Enhancer, a model-agnostic framework that can be applied to any downstream predictor. Enhancer adopts a meta-learning architecture featuring both a Temporal Meta-Learner (TML) and a Relational Meta-Learner (RML). Specifically, we introduce Reactive Point Processes Attention (RPPsAtt) within TML to overcome the limitations of missing fine-grained temporal point information, a common issue with prior methods that rely on distribution inference for mitigating temporal distribution shift. To enhance relational generalization, we introduce the Approximation-Intervention (Ant) mechanism within RML, marking the first method to mitigate relational distribution shift for quantitative investment. We conduct experiments on four long-term stock datasets, categorizing them into two tasks: stock trend prediction and stock investment recommendation. Our experimental results show that Enhancer achieves an average increase of 29.3% in profit ratio and 18.54% in the Sharpe ratio compared to the baselines across two tasks.
Machine learning models have demonstrated exceptional capabilities in solving complex problems across a variety of domains. However, these models can sometimes exhibit biased decision-making, leading to unequal treatment of different groups. Despite substantial research on counterfactual fairness, existing methods remain underdeveloped in addressing the impact of multivariate and continuous sensitive variables on decision-making outcomes. To tackle this gap, we propose a novel data pre-processing algorithm, Orthogonal to Bias (OB), which is designed to eliminate the influence of a group of continuous sensitive variables, thereby promoting counterfactual fairness in machine learning applications. Our approach, based on the assumption of an elliptical distribution within a structural causal model (SCM), shows that counterfactual fairness can be achieved by ensuring the data is orthogonal to the observed sensitive variables. The OB algorithm is model-agnostic, making it applicable to a wide range of machine learning models and tasks. To enhance numerical stability, we also introduce a sparse variant that incorporates regularization. Empirical evaluations on both simulated and real-world datasets-spanning scenarios with both discrete and continuous sensitive variables-demonstrate that our method effectively promotes fairer outcomes without compromising predictive accuracy.
Despite the theoretical expressiveness of 2-dimensional Folklore Weisfeiler-Lehman (2-FWL) Graph Neural Networks (GNNs), a significant gap persists between their theoretical capacity and their practical performance. To bridge this gap, we identify a critical limitation in current Graph Structural Encodings (GSEs): insufficient sensitivity to subtle structural variations, particularly in local connectivity, spectral features, and distance-based patterns. We show that widely used GSEs-such as Relative Random Walk Probability (RRWP) and monomial-based methods-lack full sensitivity across spectral frequency bands and long-range distances. Moreover, they fail to capture fine-grained local connectivity, which is essential for identifying cut nodes, biconnected components, and other higher-order structures that 2-FWL GNNs theoretically encode. To address these limitations, we propose CSDGSE (Connectivity, Spectral, and Distance Graph Structural Encoding), a novel GSE framework that jointly enhances sensitivity to: (1) exact local connectivity via hierarchical graph decomposition(2) full-frequency spectral features using expressive graph polynomials (e.g., Chebyshev), and (3) full-range distance interactions. A key innovation is our scalable divide-and-conquer algorithm for computing exact local connectivity across all node pairs, enabling efficient integration into modern GSEs. Extensive experiments show that CSDGSE outperforms existing GSEs in capturing complex structural patterns, achieving state-of-the-art results on molecular property prediction benchmarks like ZINC. Our work sets a new standard for GSEs by aligning theoretical expressiveness with practical effectiveness through enhanced structural sensitivity.
Machine unlearning aims to remove the information of specific data from a trained machine learning model while retaining its utility for the remaining data, so as to meet the requirements of privacy regulations. Existing unlearning methods often assume a balanced data distribution, but neglect the real-world, long-tailed scenarios, where the decision boundaries of tail classes are frequently distorted due to insufficient sample representation, thereby reducing the unlearning efficacy. In this paper, we propose the first Long-Tailed Machine Unlearning (LTMU) framework from a unified decision-boundary perspective. Our framework begins with a directional boundary repair scheme designed to enrich the distorted decision boundary of the tail class, and then develop a novel boundary retraction approach tailored for long-tailed unlearning, dispersing both the augmented and original features throughout the feature space. This bidirectional manipulation not only offers a unified interpretation of the relationship between long-tailed learning and unlearning, but also enables flexible control over both repair and unlearning processes through the generation of augmented features, thereby effectively accomplishing the long-tailed unlearning task. Extensive experiments across multiple datasets and neural network architectures demonstrate the effectiveness of our framework in achieving complete unlearning of tail classes in long-tailed distributions.
Towards Interpretable Drug-Drug Interaction Prediction: A Graph-Based Approach with Molecular and Network-Level Explanations
PDF ↗Drug-drug interactions (DDIs) represent a critical challenge in pharmacology, often leading to adverse drug reactions with significant implications for patient safety and healthcare outcomes. While graph-based methods have achieved strong predictive performance, most approaches treat drug pairs independently, overlooking the complex, context-dependent interactions unique to drug pairs. Additionally, these models struggle to integrate biological interaction networks and molecular-level structures to provide meaningful mechanistic insights. In this study, we propose MolecBioNet, a novel graph-based framework that integrates molecular and biomedical knowledge for robust and interpretable DDI prediction. By modeling drug pairs as unified entities, MolecBioNet captures both macro-level biological interactions and micro-level molecular influences, offering a comprehensive perspective on DDIs. The framework extracts local subgraphs from biomedical knowledge graphs and constructs hierarchical interaction graphs from molecular representations, leveraging classical graph neural network methods to learn multi-scale representations of drug pairs. To enhance accuracy and interpretability, MolecBioNet introduces two domain-specific pooling strategies: context-aware subgraph pooling (CASPool), which emphasizes biologically relevant entities, and attention-guided influence pooling (AGIPool), which prioritizes influential molecular substructures. The framework further employs mutual information minimization regularization to enhance information diversity during embedding fusion. Experimental results demonstrate that MolecBioNet outperforms state-of-the-art methods in DDI prediction, while ablation studies and embedding visualizations further validate the advantages of unified drug pair modeling and multi-scale knowledge integration. Moreover, the pooling strategies yield interpretable insights into the underlying interaction mechanisms, highlighting MolecBioNet's potential as a robust and transparent framework for predictive modeling in support of drug safety assessment.
Large language models (LLMs) have demonstrated their strong capabilities in various domains, and have been recently integrated for graph analysis as graph language models (GLMs). With LLMs as the predictor, some GLMs can interpret unseen tasks described by natural language, and learn from a few examples in the prompts without parameter tuning, known as in-context learning (ICL). Another subset of GLMs utilizes abundant training labels to enhance model performance, known as instruction tuning. However, we argue that ICL on graphs has effectiveness issues due to fixed parameters and efficiency issues due to long context. Meanwhile, the large amount of labeled data required for instruction tuning can be difficult to obtain in real-world scenarios. To this end, we aim to introduce an extra parameter adaptation stage that can efficiently tailor GLMs to an unseen graph and task with only a few labeled examples, in exchange for better prediction accuracy and faster inference speed. For implementation, in this paper we propose GraphLAMA method, with its model backbone and learning schemes specialized for efficient tuning and inference. Specifically, for the model backbone, we use a graph neural network (GNN) with several well-designed components (e.g., hop encodings, gating modules) to transform nodes into the representation space of LLM tokens. Task instructions can then be represented as a mixture of node and language tokens. In the pre-training stage, all model parameters except for the LLM will be trained with different tasks (i.e., node matching, node classification, and link prediction) to capture general knowledge. In the adaptation stage, only a few pre-trained parameters will be updated based on few-shot examples. Extensive experiments on few/zero-shot node classification and summary generation show that our proposed GraphLAMA achieves state-of-the-art (SOTA) performance with 4.91% absolute improvement in accuracy. Compared with ICL, our inference speed can be 10 times faster under 5-shot setting. Our code is available on GitHub at https://github.com/BUPT-GAMMA/GraphLAMA.
Graph anomaly detection (GAD) is a critical task with applications in domains such as networking, finance, and bioinformatics. % However, the scarcity of labeled anomalies and the limitations of unsupervised methods hinder effective detection. % While semi-supervised and few-shot learning approaches offer improvements, they struggle with knowledge transfer and rely heavily on labeled data. % Recent advancements in prompt tuning on graphs provide a promising direction, but their application to heterophilous graphs in anomaly detection remains underexplored. % In this work, we propose AffinityTune, a novel framework for few-shot graph anomaly detection based on prompt tuning. % Our approach introduces a unified task framework grounded in affinity judgment, incorporating multi-granularity tasks (''node vs. adjacent subgraph'' and ''node vs. community'') to learn diverse anomaly priors. % For downstream tasks, we reformulate binary classification into a ''node vs. global representation'' affinity judgment, mapping anomalies to a hypersphere in feature space. % We also design learnable task heads to synergize multiple tasks and propose FlexPrompt, a flexible prompt-tuning strategy for fine-grained adaptation to downstream tasks. % Extensive experiments on real-world and synthetic datasets demonstrate that AffinityTune significantly outperforms existing methods in detection efficacy. % Our contributions include a unified task framework, multi-granularity tasks, and the FlexPrompt strategy, offering a parameter-efficient and adaptable solution for few-shot graph anomaly detection.
Decoupling Spatio-Temporal Prediction: When Lightweight Large Models Meet Adaptive Hypergraphs
PDF ↗Spatio-temporal prediction is a pivotal task with broad applications in traffic management, climate monitoring, energy scheduling, etc. However, existing methodologies often struggle to balance model expressiveness and computational efficiency, especially when scaling to large real-world datasets. To tackle these challenges, we propose STH-SepNet (Spatio-Temporal Hypergraph Separation Networks), a novel framework that decouples temporal and spatial modeling to enhance both efficiency and precision. Therein, the temporal dimension is modeled using lightweight large language models, which effectively capture low-rank temporal dynamics. Concurrently, the spatial dimension is addressed through an adaptive hypergraph neural network, which dynamically constructs hyperedges to model intricate, higher-order interactions. A carefully designed gating mechanism is integrated to seamlessly fuse temporal and spatial representations. By leveraging the fundamental principles of low-rank temporal dynamics and spatial interactions, STH-SepNet offers a pragmatic and scalable solution for spatio-temporal prediction in real-world applications. Extensive experiments on large-scale real-world datasets across multiple benchmarks demonstrate the effectiveness of STH-SepNet in boosting predictive performance while maintaining computational efficiency. This work may provide a promising lightweight framework for spatio-temporal prediction, aiming to reduce computational demands and while enhancing predictive performance. Our code is avaliable at https://github.com/SEU-WENJIA/ST-SepNet-Lightweight-LLMs-Meet-Adaptive-Hypergraphs.
Unlocking the Power of Diffusion Models in Sequential Recommendation: A Simple and Effective Approach
PDF ↗In this paper, we focus on the often-overlooked issue of embedding collapse in existing diffusion-based sequential recommendation models and propose ADRec, an innovative framework designed to mitigate this problem. Diverging from previous diffusion-based methods, ADRec applies an independent noise process to each token and performs diffusion across the entire target sequence during training. ADRec captures token interdependency through auto-regression while modeling per-token distributions through token-level diffusion. This dual approach enables the model to effectively capture both sequence dynamics and item representations, overcoming the limitations of existing methods. To further mitigate embedding collapse, we propose a three-stage training strategy: (1) pre-training the embedding weights(2) aligning these weights with the ADRec backbone, and (3) fine-tuning the model. During inference, ADRec applies the denoising process only to the last token, ensuring that the meaningful patterns in historical interactions are preserved. Our comprehensive empirical evaluation across six datasets underscores the effectiveness of ADRec in enhancing both the accuracy and efficiency of diffusion-based sequential recommendation systems.
Spatio-temporal forecasting is crucial in various fields and requires a careful balance between identifying subtle patterns and filtering out noise. Vector quantization (VQ) appears well-suited for this purpose, as it quantizes input vectors into a set of codebook vectors or patterns. Although VQ has shown promise in various computer vision tasks, it surprisingly falls short in enhancing the accuracy of spatio-temporal forecasting. We attribute this to two main issues: inaccurate optimization due to non-differentiability and limited representation power caused by hard-VQ structure. To tackle these challenges, we introduce Differentiable Sparse Soft-Vector Quantization (SVQ), the first VQ method to enhance spatio-temporal forecasting. SVQ balances detail preservation with noise reduction, offering full differentiability and a solid foundation in sparse regression. The method employs a two-layer MLP and an extensive codebook to streamline the sparse regression process, significantly cutting computational costs while simplifying training and improving performance. Empirical studies on five spatio-temporal benchmark datasets show SVQ achieves state-of-the-art results, including a 7.9% improvement on the WeatherBench-S temperature dataset and an average mean absolute error reduction of 9.4% in video prediction benchmarks (Human3.6M, KTH, and KittiCaltech), along with a 17.3% enhancement in image quality (measured by LPIPS).
Machine learning models in real-world applications often suffer performance issues due to data distribution shifts. Temporal domain generalization aims to adapt models to the ''concept drift,'' maintaining future performance. Existing works based on model-centric training strategies may entail extensive interaction between data and model to appropriately train the model for distribution shifts. To this end, we aim to nip the problem in the bud by generating future domain data for model training and naturally bypassing the cumbersome interaction between data and model. We propose the COncept Drift simulAtor (CODA) framework incorporating a predicted feature correlation matrix to simulate future data for model training. Specifically, the feature correlations matrix serves as a delegation to represent data characteristics at each time point and the trigger for future data generation. Experimental results demonstrate that using CODA-generated data as training input effectively achieves temporal domain generalization across different model architectures with great transferability.