论文检索

Zhiyuan Su, Sunhao Dai, Xiao Zhang 0034

Clustering of Bandits (CB) methods enhance sequential decision-making by grouping bandits into clusters based on similarity and incorporating cluster-level contextual information, demonstrating effectiveness and adaptability in applications like personalized streaming recommendations. However, when extending CB algorithms to their neural version (commonly referred to as Clustering of Neural Bandits, or CNB), they suffer from loss of plasticity, where neural network parameters become rigid and less adaptable over time, limiting their ability to adapt to non-stationary environments (e.g., dynamic user preferences in recommendation). To address this challenge, we propose Selective Reinitialization (SeRe), a novel bandit learning framework that dynamically preserves the adaptability of CNB algorithms in evolving environments. SeRe leverages a contribution utility metric to identify and selectively reset underutilized units, mitigating loss of plasticity while maintaining stable knowledge retention. Furthermore, when combining SeRe with CNB algorithms, the adaptive change detection mechanism adjusts the reinitialization frequency according to the degree of non-stationarity, ensuring effective adaptation without unnecessary resets. Theoretically, we prove that SeRe enables sublinear cumulative regret in piecewise-stationary environments, outperforming traditional CNB approaches in long-term performances. Extensive experiments on six real-world recommendation datasets demonstrate that SeRe-enhanced CNB algorithms can effectively mitigate the loss of plasticity with lower regrets, improving adaptability and robustness in dynamic settings.

Zhixiang Su, Di Wang 0004, Chunyan Miao

Recent investigations on the effectiveness of Graph Neural Network (GNN)-based models for link prediction in Knowledge Graphs (KGs) show that vanilla aggregation does not significantly impact the model performance. In this paper, we introduce a novel method, named Context Pooling, to enhance GNN-based models' efficacy for link predictions in KGs. To our best of knowledge, Context Pooling is the first methodology that applies graph pooling in KGs. Additionally, Context Pooling is first-of-its-kind to enable the generation of query-specific graphs for inductive settings, where testing entities are unseen during training. Specifically, we devise two metrics, namely neighborhood precision and neighborhood recall, to assess the neighbors' logical relevance regarding the given queries, thereby enabling the subsequent comprehensive identification of only the logically relevant neighbors for link prediction. Our method is generic and assessed by being applied to two state-of-the-art (SOTA) models on three public transductive and inductive datasets, achieving SOTA performance in 42 out of 48 settings.

Jiajie Su, Chaochao Chen 0001, Weiming Liu 0005, Yuhang Wang, Yihao Wang 0007, Zheqi Lv, Xiaolin Zheng, Jianwei Yin

Session-based Recommendation (SBR) explores dynamic user interests based on short anonymous sessions. The performance of traditional ID-collaborative SBR models is constrained by the limited session length. To this end, we focus on Multi-modal Session-based Recommendation (MMSBR), which aims to leverage modality knowledge to promote short-term user interest modeling. Previous studies cannot solve the MMSBR problem well due to issues as: (I1) Modal-invariant and specific dependencies are difficult to disentangle and unify. (I2) Multiple modal transitions produce mutual effects. (I3) Modal-aware personalized preference discrepancy exists. Thus, we propose a modality-aware diffusion-based framework MDSD with subspace disentanglement, which decouples multi-modal collaborations and provides a new paradigm that integrates modality representation generation into next-item prediction. We first employ modality-disentangled consistency graphs based on affinity semantics to extract the unified modal subspace. Then we design a cross-modal contrastive attention fusion that explores cross-modality interplay, enhancing modal alignment and consistency. To interpret personalized interest discrepancy, we realize the preference-guided multi-modal diffusion, which combines modality-aware interest generation with prediction based on collaborative signals, thus providing comprehensive recommendation results. Extensive experiments on real-world datasets demonstrate the effectiveness of MDSD.

Will Stamey, Sriram Somanchi, Edward McFowland III

Difference-in-differences (DiD) has been extensively applied in the literature to elicit the average causal effect of an intervention or policy. Though researchers explore heterogeneity in the treatment effect with respect to time or some observed covariate (usually driven by domain knowledge), there is limited work on a principled, algorithmic approach to discovering the subgroups that exhibit heterogeneity in the treatment effect in DiD settings. In this research, we propose the Difference-in-Differences Subset Scan (DiD-Scan), which finds subregions of the observed covariate space corresponding to anomalous patterns in the conditional mean treatment effect. We mold our method to the DiD setting by also enabling a scan for dynamic effects over time and across multiple outcome variables. This supports the discovery of patterns where a subset of outcomes are highly impacted for a limited time window. We develop a generalized likelihood ratio-based scoring function to quantify the treatment effect of a given subgroup and propose a computationally efficient method to discover the subgroups with the largest treatment effect. We extend the method to consider correlations across time, a common condition in difference-in-difference settings that increases the difficulty of subset identification. Lastly, we demonstrate the efficacy and interpretability of the method with both simulations and applications to real datasets, replicating and extending two published studies.

Zixing Song, Muzhi Li 0001, Yifei Zhang, Irwin King, José Miguel Hernández-Lobato

The prevalence of temporal networks in real-world applications, like financial transaction networks for loan approval prediction, poses significant challenges for ensuring fairness across different groups. These dynamic systems increasingly rely on Temporal Graph Neural Networks (TGNNs) to model evolving interactions between users over time, but TGNNs can inadvertently produce unfair outcomes across different demographic groups. In this work, we are the first to investigate group fairness on temporal graphs and propose a novel real-time framework for monitoring and improving group fairness in TGNNs. We begin by incorporating a fixed fairness regularization term into the TGNN framework, named FTGNN-R, which operates in real-time but exhibits several critical limitations. To address this, we propose FTGNN-M, a new monitoring-based approach that assesses fairness on the fly, without relying on unseen test data. By conducting a sensitivity analysis, FTGNN-M further identifies the specific channels of node embeddings responsible for unfairness and adaptively adjusts the corresponding subset of model parameters. This approach enables a trade-off between fairness and utility in dynamic settings. FTGNN-M offers theoretical guarantees for both fairness assessment and fairness promotion. Extensive experiments on five temporal transaction network datasets demonstrate the effectiveness of our proposed FTGNN-M model in terms of both utility and fairness metrics.

Zhao Song 0002, Chongxi Wang, Guangyi Xu 0002, Junze Yin

Large language models (LLMs) have significantly improved various aspects of our daily lives. These models have impacted numerous domains, from healthcare to education, enhancing productivity, decision-making processes, and accessibility. However, the quadratic complexity of attention in transformer architectures makes it impractical to train very large models on lengthy texts or use them efficiently during inference. While a recent study by [Kacham, Mirrokni and Zhong 2023] introduced a technique that replaces the softmax with a polynomial function and polynomial sketching to speed up attention mechanisms, the theoretical understandings of this new approach are not yet well understood. In this paper, we offer a theoretical analysis of the expressive capabilities of polynomial attention. Our study reveals a disparity in the ability of high-degree and low-degree polynomial attention. Specifically, we construct two carefully designed datasets, namely D 0 and D 1, where D 1 includes a feature with a significantly larger value compared to D 0. We demonstrate that with a sufficiently high degree β, a single-layer polynomial attention network can distinguish between D 0 and D 1. However, with a low degree β, the network cannot effectively separate the two datasets. This analysis underscores the greater effectiveness of high-degree polynomials in amplifying large values and distinguishing between datasets. Our analysis offers insight into the representational capacity of polynomial attention and provides a rationale for incorporating higher-degree polynomials in attention mechanisms to capture intricate linguistic correlations.

Yu Song 0007, Zhigang Hua, Harry Shomer, Yan Xie, Jingzhe Liu, Bo Long, Hui Liu 0031

Link Prediction (LP) is a critical task in graph machine learning. While Graph Neural Networks (GNNs) have significantly advanced LP performance recently, existing methods face key challenges including limited supervision from sparse connectivity, sensitivity to initialization, and poor generalization under distribution shifts. We explore pretraining as a solution to address these challenges. Unlike node classification, LP is inherently a pairwise task, which requires the integration of both node- and edge-level information. In this work, we present the first systematic study on the transferability of these distinct modules and propose a late fusion strategy to effectively combine their outputs for improved performance. To handle the diversity of pretraining data and avoid negative transfer, we introduce a Mixture-of-Experts (MoE) framework that captures distinct patterns in separate experts, facilitating seamless application of the pretrained model on diverse downstream datasets. For fast adaptation, we develop a parameter-efficient tuning strategy that allows the pretrained model to adapt to unseen datasets with minimal computational overhead. Experiments on 16 datasets across two domains demonstrate the effectiveness of our approach, achieving state-of-the-art performance on low-resource link prediction while obtaining competitive results compared to end-to-end trained methods, with over 10,000x lower computational overhead.

Wenhao Song, Xuan Wu 0004, Bo Yang 0002, You Zhou 0008, Yubin Xiao, Yanchun Liang 0001, Hongwei Ge, Heow Pueh Lee, Chunguo Wu

To address the weight coupling problem, certain studies introduced few-shot Neural Architecture Search (NAS) methods, which partition the supernet into multiple sub-supernets. However, these methods often suffer from computational inefficiency and tend to provide suboptimal partitioning schemes. To address this problem more effectively, we analyze the weight coupling problem from a novel perspective, which primarily stems from distinct modules in succeeding layers imposing conflicting gradient directions on the preceding layer modules. Based on this perspective, we propose the Gradient Contribution (GC) method that efficiently computes the cosine similarity of gradient directions among modules by decomposing the Vector-Jacobian Product during supernet backpropagation. Subsequently, the modules with conflicting gradient directions are allocated to distinct sub-supernets while similar ones are grouped together. To assess the advantages of GC and address the limitations of existing Graph Neural Architecture Search methods, which are limited to searching a single type of Graph Neural Networks (Message Passing Neural Networks (MPNNs) or Graph Transformers (GTs)), we propose the Unified Graph Neural Architecture Search (UGAS) framework, which explores optimal combinations of MPNNs and GTs. The experimental results demonstrate that GC achieves state-of-the-art (SOTA) performance in supernet partitioning quality and time efficiency. In addition, the architectures searched by UGAS+GC outperform both the manually designed GNNs and those obtained by existing NAS methods. Finally, ablation studies further demonstrate the effectiveness of all proposed methods.

Patrick Soga, Zhenyu Lei 0004, Yinhan He, Camille L. Bilodeau, Jundong Li

Predicting changes in binding free energy (ΔΔ G) is a vital task in protein engineering and protein-protein interaction (PPI) engineering for drug discovery. Previous works have observed a high correlation between ΔΔ G and entropy, using probabilities of biologically important objects such as side chain angles and residue identities to estimate ΔΔ G. However, estimating the full conformational distribution of a protein complex is generally considered intractable. In this work, we propose a new approach to ΔΔ G prediction that avoids this issue by instead leveraging energy-based models for estimating the probability of a complex's conformation. Specifically, we novelly decompose ΔΔ G into a sequence-based component estimated by an inverse folding model and a structure-based component estimated by an energy model. This decomposition is made tractable by assuming equilibrium between the bound and unbound states, allowing us to simplify the estimation of degeneracies associated with each state. Unlike previous deep learning-based methods, our method incorporates an energy-based physical inductive bias by connecting the often-used sequence log-odds ratio-based approach to ΔΔ G prediction with a new ΔΔ E term grounded in statistical mechanics. We demonstrate superiority over existing state-of-the-art structure and sequence-based deep learning methods in ΔΔ G prediction and antibody optimization against SARS-CoV-2.

Hongzhe Shou, Guanyu Lu, Martin Pavlovski, Fang Zhou

Existing anomaly detection methods tend to utilize a large amount of training data to learn patterns of normal data for effective anomaly identification, but such methods typically incur substantial training time overhead. Considering that unlabeled data often contains a lot of redundant information, selecting and utilizing a small yet representative subset instead of the entire dataset can significantly improve training efficiency while maintaining detection performance. To this end, we introduce an end-to-end reinforcement learning framework with a balanced sampling strategy that targets both normal and abnormal instances. This framework identifies and exploits potential anomalies in the unlabeled data while sampling peripheral normal instances (often difficult to detect), thereby enhancing the overall anomaly detection performance without requiring excessive time for the sampling process. Additionally, we present a joint reward mechanism, combined with inconsistency penalties, which optimizes both an agent's action space and the representation space, ultimately improving the quality of the sampling process. Extensive experiments on four public datasets from different domains demonstrate the effectiveness and efficiency of our framework. The code is available at https://github.com/ZhouF-ECNU/READ.

Yujin Shin, Taeyoung Yun, Sujin Yun, Sungpil Woo, Sunhwan Lim, Jinkyoo Park

Wind farms generate electricity from wind, offering a sustainable and eco-friendly power source. In wind farms, determining the positions of wind turbines is crucial for high energy production due to complex wake interactions. However, optimizing wind farm layouts for given wind conditions remains challenging. While exist- ing methods try to solve the problem by training a surrogate model and employing optimization algorithms based on the model, these methods require a large number of simulations to obtain highly productive layouts, which is time-consuming in large-scale tasks. Furthermore, they mostly yield less diverse layouts, making the deployment of such layouts into real-world scenarios difficult. To address these challenges, we introduce a novel conditional genera- tive modeling approach to find wind farm layouts that maximize energy production. Our method consists of four stages. First, we collect datasets that consist of layouts and their corresponding an- nual energy production (AEP). Then, we train a diffusion model conditioned on AEP and wind scenarios using the collected dataset. Specifically, we use a Graph Neural Network as the backbone for the denoising network to ensure permutation invariance. Next, we sample promising layouts from the trained diffusion model by conditioning with a high AEP and given wind scenario. We also in- troduce a local search algorithm, which enables us to adjust layouts that violate design constraints. Finally, we evaluate the generated layouts and augment the dataset. We repeat these processes itera- tively to further optimize the layouts. Throughout these processes, we effectively generate highly productive and diverse wind farm layouts in a sample-efficient manner. Experimental results demon- strate that our approach outperforms state-of-the-art methods in maximizing energy production across various scenarios.

Qilong Shi, Wei Zhou 0077, Yizhuo Zheng, Xinye Xu, Ting Yang, Yuanyuan Zhang 0006, Long Yao, Yangyang Wang 0001, Mingwei Xu 0001

Quantile estimation is critical for diverse applications, including database management and network traffic monitoring. Probabilistic quantile sketches are widely employed in practice, with the KLL sketch (introduced in 2016) being particularly notable for its theoretically space-optimal properties. However, KLL overlooks the inherent repetition of elements often present in real-world data streams. Such streams are frequently highly skewed, characterized by ''hot items''-items that appear with high frequency. The KLL sketch processes these hot items without accounting for their prevalence, resulting in suboptimal space utilization due to redundant insertions and storage. To overcome this limitation, we propose Cooled-KLL, an enhanced KLL sketch. Cooled-KLL introduces a novel ''Hot Filter'' structure that efficiently identifies and stores hot items as compact key-value pairs. This mechanism ensures that only ''cold'' (less frequent) items are subsequently processed by the core KLL sketch. Our approach significantly reduces memory consumption without compromising processing speed. Extensive experiments demonstrate that Cooled-KLL consistently outperforms five other state-of-the-art algorithms, achieving up to 2.5 orders of magnitude higher accuracy compared to the standard KLL sketch.

Liangliang Shi, Haoran Zhang, Shuheng Shen, Changhua Meng, Weiqiang Wang 0002, Junchi Yan

The Quadratic Assignment Problem (QAP) has attracted lasting attention for its wide applications and computational challenges. Despite the rich literature in machine learning for QAP, most works often address the problem in the setting of image matching, whereby deep networks could play a vital role in extracting useful features for the subsequent matching. While its power on pure numerical QAP instances is limited in node embedding, often with a vanilla graph neural network. This paper tries to tap the potential of deep nets for QAP, specifically by modifying the input instance, which is orthogonal to previous efforts. Specifically, we develop a bi-level unsupervised framework, where the inner optimization involves trying to solve the modified instance with entropic regularization that can be solved iteratively using the Sinkhorn algorithm without affecting backpropagation by truncating gradients during training. The outer minimization deals with the quadratic objective function of the original QAP. In particular, seeing the intractable scale of the most general form, i.e., Lawler's QAP and the practical utility of the more efficient Koopmans-Beckmann QAP (KBQAP) form for solving other graph and combinatorial problems like TSP and graph edit distance, we embody our network on the KBQAP, and show its strong performance on various benchmarks in our experiments.

Chengzhi Shi, Stratis Ioannidis

Survival analysis is widely deployed in a diverse set of fields, including healthcare, business, ecology, etc. The Cox Proportional Hazard (CoxPH) model is a semi-parametric model often encountered in the literature. Despite its popularity, wide deployment, and numerous variants, scaling CoxPH to large datasets and deep architectures poses a challenge, especially in the high-dimensional regime. We identify a fundamental connection between rank regression and the CoxPH model: this allows us to adapt and extend the so-called spectral method for rank regression to survival analysis. Our approach is versatile, naturally generalizing to several CoxPH variants, including deep models. We empirically verify our method's scalability on multiple real-world high-dimensional datasets; our method outperforms legacy methods w.r.t. predictive performance and efficiency.

Zeang Sheng, Weiyang Guo, Yingxia Shao, Wentao Zhang 0001, Bin Cui 0001

Graph Neural Networks (GNNs) have achieved significant success in various graph learning tasks. However, their success relies heavily on the availability of adequate high-quality labels, which requires high annotation costs. Recently, Large Language Models (LLMs) became known to the community for their strong zero-shot performance on diverse textual tasks. To utilize LLM's zero-shot abilities, existing work substitutes the oracle in the graph active learning setting with an LLM and achieves low-cost annotation compared to consulting human experts. However, annotations from LLMs inevitably contain noise, and we find through an empirical analysis that LLMs possess complex and dataset-specific annotation noise distributions. This intricacy of LLMs' annotation noise is neglected by existing work, leading to sub-optimal annotation data quality under the same budget. In this paper, we propose a Dataset- and LLM-Aware graph active learning framework called DMA that explicitly models LLM's annotation noise distributions on the given dataset. Concretely, DMA queries the oracle LLM with manually constructed prompts for each class's pseudo samples that are then used to measure the class-wise annotation noise rates. We then equip DMA with a new noise-aware graph active learning algorithm to utilize this fine-grained annotation noise distribution. Extensive performance optimizations are applied to the implementation to boost DMA's scalability and runtime efficiency on large-scale datasets. Evaluations on five text-attributed graph datasets show that DMA consistently outperforms all the baselines in terms of annotation data quality, which illustrates the importance of careful handling of LLM's intricate annotation noise.

Akshit Sharma, Sam Reinehr, Dinesh Mehta, Bo Wu 0002

Frequent Subgraph Mining (FSM) is the process of identifying common subgraph patterns that occur over a certain threshold. The NP-hardness of FSM makes it a complex and time-consuming task. FSM is generally solved in 2 steps, 1). Generation step: determining the possible patterns that can be frequent and 2). Metric step: determining if the pattern is frequent. In the literature, the generation step is usually solved by vertex or edge extension methods. These methods produce a lot of redundant candidate patterns which must be removed, and hence increasing the latency. To address these challenges, the paper introduces a vertex based merging method which reduces the redundancies, and introduces effective pruning mechanisms. Moreover, the existing metrics to determine if a pattern is frequent or not, either is highly accurate while demanding significant computational time (Maximum Independent Set (MIS)) or overestimates the pattern count while taking less time (Minimum Node Image (MNI)). Thus, the paper introduces ''Maximal Independent Set (mIS)'' metric, which minimizes latency while obtaining accuracy close to MIS, which is controlled by a pattern overlap parameter (łambda).Through extensive experimentation, our proposed method achieves an average of 10.58× speedup when compared to GraMi and an average of 3× speedup when compared to T-FSM.

Zezhi Shao, Yujie Li 0008, Fei Wang 0014, Chengqing Yu, Yisong Fu, Tangwen Qian, Bin Xu 0019, Boyu Diao, Yongjun Xu 0001, Xueqi Cheng 0001

The advent of universal time series forecasting models has revolutionized zero-shot forecasting across diverse domains, yet the critical role of data diversity in training these models remains underexplored. Existing large-scale time series datasets often suffer from inherent biases and imbalanced distributions, leading to suboptimal model performance and generalization. To address this gap, we introduce BLAST, a novel pre-training corpus designed to enhance data diversity through a balanced sampling strategy. First, BLAST incorporates 321 billion observations from publicly available datasets and employs a comprehensive suite of statistical metrics to characterize time series patterns. Then, to facilitate pattern-oriented sampling, the data is implicitly clustered using grid-based partitioning. Furthermore, by integrating grid sampling and grid mixup techniques, BLAST ensures a balanced and representative coverage of diverse patterns. Experimental results demonstrate that models pre-trained on BLAST achieve state-of-the-art performance with a fraction of the computational resources and training tokens required by existing methods. Our findings highlight the pivotal role of data diversity in improving both training efficiency and model performance for the universal forecasting task.

Wei Shao, Ziquan Fang, Lu Chen 0001, Yunjun Gao

Trajectory anomaly detection aims to identify patterns in trajectory data that deviate significantly from normal behavior, such as taxi detours, and plays a crucial role in urban computing. However, real-world trajectories are inherently complex, containing diverse anomalies and unavoidable noise. Existing research mainly focuses on coarse-grained trajectory anomalies, such as detour and switch anomalies, while paying limited attention to fine-grained trajectory anomalies, such as time and loop anomalies. Furthermore, they tend to disregard the impact of inherent noise in trajectories. As a result, there remains a gap in developing robust models with strong generalization capabilities to effectively detect fine-grained trajectory anomalies, even in noisy environments. Inspired by the powerful generalization and noise resistance capabilities of LLMs, we propose an LLM-driven approach, FOTraj, for fine-grained and noise-resilient trajectory anomaly detection. Specifically, FOTraj introduces a Spatio-Temporal Dependency Encoder (STE) that converts raw trajectory data into a spatio-temporal graph, equipped with a tailored adapter for the LLM to capture finer-grained trajectory features. Besides, we design a Patch-Gated Robustification (PGR) module to simulate noisy environments, enhancing the LLM's resilience to trajectory noise. Since LLMs were originally designed for sequence modeling and cannot directly process graph structured data, FOTraj incorporates a specialized graph decoder to map the LLM's output back into a graph structure, utilizing a temperature-controlled top-k sampling mechanism to improve diversity and robustness. Extensive experiments on three datasets show that FOTraj outperforms state-of-the-art methods.

Timur Sattarov, Marco Schreyer, Damian Borth

Anomaly detection in tabular data remains challenging due to complex feature interactions and the scarcity of anomalous examples. Denoising autoencoders rely on fixed-magnitude noise, limiting adaptability to diverse data distributions. Diffusion models introduce scheduled noise and iterative denoising, but lack explicit reconstruction mappings. We propose the Diffusion-Scheduled Denoising Autoencoder (DDAE), a framework that integrates diffusion-based noise scheduling and contrastive learning into the encoding process to improve anomaly detection. We evaluated DDAE on 57 datasets from ADBench. Our method outperforms in semi-supervised settings and achieves competitive results in unsupervised settings, improving PR-AUC by up to 65% (9%) and ROC-AUC by 16% (6%) over state-of-the-art autoencoder (diffusion) model baselines. We observed that higher noise levels benefit unsupervised training, while lower noise with linear scheduling is optimal in semi-supervised settings. These findings underscore the importance of principled noise strategies in tabular anomaly detection.

Rabeya Tus Sadia, Md. Atik Ahamed, Qiang Cheng 0001

The integration of single-cell RNA sequencing (scRNA-seq) and spatial transcriptomics (ST) data is crucial for understanding gene expression in spatial context. Existing methods for such integration have limited performance, with structural similarity often below 60%, We attribute this limitation to the failure to consider causal relationships between genes. We present CausalGeD, which combines diffusion and autoregressive processes to leverage these relationships. By generalizing the Causal Attention Transformer from image generation to gene expression data, our model captures regulatory mechanisms without predefined relationships. Across 10 tissue datasets, CausalGeD outperforms state-of-the-art baselines by 5- 32% in key metrics, including Pearson's correlation and structural similarity, advancing both technical and biological insights.