论文检索

Xinyu Lin 0001, Wenjie Wang 0007, Yongqi Li 0001, Fuli Feng, See-Kiong Ng, Tat-Seng Chua

Harnessing Large Language Models (LLMs) for recommendation is rapidly emerging, which relies on two fundamental steps to bridge the recommendation item space and the language space: 1) item indexing utilizes identifiers to represent items in the language space, and 2) generation grounding associates LLMs' generated token sequences to in-corpus items. However, previous methods exhibit inherent limitations in the two steps. Existing ID-based identifiers (e.g., numeric IDs) and description-based identifiers (e.g., titles) either lose semantics or lack adequate distinctiveness. Moreover, prior generation grounding methods might generate invalid identifiers, thus misaligning with in-corpus items. To address these issues, we propose a novel Transition paradigm for LLM-based Recommender (named TransRec) to bridge items and language. Specifically, TransRec presents multi-facet identifiers, which simultaneously incorporate ID, title, and attribute for item indexing to pursue both distinctiveness and semantics. Additionally, we introduce a specialized data structure for TransRec to ensure generating valid identifiers only and utilize substring indexing to encourage LLMs to generate from any position of identifiers. Lastly, TransRec presents an aggregated grounding module to leverage generated multi-facet identifiers to rank in-corpus items efficiently. We instantiate TransRec on two backbone models, BART-large and LLaMA-7B.

Qilong Lin, Zhenzhe Zheng 0001, Fan Wu 0006

In online advertising, existing auto-bidding strategies for bid shading mainly adopt the approach of first predicting the winning price distribution and then calculating the optimal bid. However, the winning price information available to the Demand Side Platforms (DSPs) is extremely limited, and the associated uncertainties make it challenging for DSPs to accurately estimate winning price distribution. To address this challenge, we conducted a comprehensive analysis of the process by which DSPs obtain winning price information, and abstracted two types of uncertainties from it: known uncertainty and unknown uncertainty. Based on these uncertainties, we proposed two levels of robust bidding strategies: Robust Bidding for Censorship (RBC) and Robust Bidding for Distribution Shift (RBDS), which offer guarantees for the surplus in the worst-case scenarios under uncertain conditions. Experimental results on public datasets demonstrate that our robust bidding strategies consistently enable DSPs to achieve superior surpluses, both on test sets and under worst-case conditions.

Longlong Lin, Tao Jia 0001, Zeli Wang, Jin Zhao 0003, Rong-Hua Li 0001

Higher-order graph clustering aims to partition the graph using frequently occurring subgraphs (i.e., motifs), instead of the lower-order edges, as the atomic clustering unit, which has been recognized as the state-of-the-art solution in ground truth community detection and knowledge discovery. Motif conductance is one of the most promising higher-order graph clustering models due to its strong interpretability. However, existing motif conductance based graph clustering algorithms are mainly limited by a seminal two-stage reweighting computing framework, needing to enumerate all motif instances to obtain an edge-weighted graph for partitioning. However, such a framework has two-fold vital defects: (1) It can only provide a quadratic bound for the motif with three vertices, and whether there is provable clustering quality for other motifs is still an open question. (2) The enumeration procedure of motif instances incurs prohibitively high costs against large motifs or large dense graphs due to combinatorial explosions. Besides, expensive spectral clustering or local graph diffusion on the edge-weighted graph also makes existing methods unable to handle massive graphs with millions of nodes. To overcome these dilemmas, we propose a Provable and Scalable Motif Conductance algorithm PSMC, which has a fixed and motif-independent approximation ratio for any motif. Specifically, PSMC first defines a new vertex metric Motif Resident based on the given motif, which can be computed locally. Then, it iteratively deletes the vertex with the smallest motif resident value very efficiently using novel dynamic update technologies. Finally, it outputs the locally optimal result during the above iterative process. To further boost efficiency, we propose several effective bounds to estimate the motif resident value of each vertex, which can greatly reduce computational costs. Empirical results on real-life and synthetic demonstrate that our proposed algorithms achieve 3.2-32 times speedup and improve the quality by at least 12 times than the state-of-the art baselines.

Li Lin 0011, Zhiqiang Lu, Shuai Wang 0008, Yunhuai Liu, Zhiqing Hong, Haotian Wang 0008, Shuai Wang 0008

Recently, integrated warehouse and distribution logistics systems are widely used in E-commerce industries to adjust to constantly changing customer demands. It makes the prediction of purchase demand and delivery supply capacity a crucial problem to streamline operations and improve efficiency. The interaction between such demand and supply not only relies on their economic relationships but also on consumer psychology caused by daily events, such as epidemics, promotions, and festivals. Although existing studies have made great efforts in the joint prediction of demand and supply considering modeling the demand-supply interactions, they seldom refer to the impacts of diverse events. In this work, we propose MulSTE, a Multi-view Spatio-Temporal learning framework with heterogeneous Event fusion. Firstly, an Event Fusion Representation (EFR) module is designed to fuse the textual, numerical, and categorical heterogeneous information for emergent and periodic events. Secondly, a Multi-graph Adaptive Convolution Recurrent Network (MGACRN) is developed as the spatio-temporal encoder (ST-Encoder) to capture the evolutional features of demand, supply, and events. Thirdly, the Event Gated Demand-Supply Interaction Attention (EGIA) module is designed to model the demand-supply interactions during events. The evaluations are conducted on two real-world datasets collected from JD Logistics and public websites. The experimental results show that our method outperforms state-of-the-art baselines in various metrics.

Fake Lin, Ziwei Zhao 0002, Xi Zhu 0004, Da Zhang, Shitian Shen, Xueying Li 0004, Tong Xu 0001, Suojuan Zhang, Enhong Chen

Last year has witnessed the re-flourishment of tag-aware recommender systems supported by the LLM-enriched tags. Unfortunately, though large efforts have been made, current solutions may fail to describe the diversity and uncertainty inherent in user preferences with only tag-driven profiles. Recently, with the development of geometry-based techniques, e.g., box embeddings, the diversity of user preferences now could be fully modeled as the range within a box in high dimension space. However, defect still exists as these approaches are incapable of capturing high-order neighbor signals, i.e., semantic-rich multi-hop relations within the user-tag-item tripartite graph, which severely limits the effectiveness of user modeling. To deal with this challenge, in this paper, we propose a novel framework, called BoxGNN, to perform message aggregation via combinations of logical operations, thereby incorporating high-order signals. Specifically, we first embed users, items, and tags as hyper-boxes rather than simple points in the representation space, and define two logical operations, i.e., union and intersection, to facilitate the subsequent process. Next, we perform the message aggregation mechanism via the combination of logical operations, to obtain the corresponding high-order box representations. Finally, we adopt a volume-based learning objective with Gumbel smoothing techniques to refine the representation of boxes. Extensive experiments on two publicly available datasets and one LLM-enhanced e-commerce dataset have validated the superiority of BoxGNN compared with various state-of-the-art baselines. The code is released online: https://github.com/critical88/BoxGNN.

Zukang Liao, Min Chen 0001

Image similarity has been extensively studied in computer vision. In recent years, machine-learned models have shown their ability to encode more semantics than traditional multivariate metrics. However, in labelling semantic similarity, assigning a numerical score to a pair of images is impractical, making the improvement and comparisons on the task difficult. In this work, we present a more intuitive approach to build and compare image similarity models based on labelled data in the form of A:R vs B:R, i.e., determining if an image A is closer to a reference image R than another image B. We address the challenges of sparse sampling in the image space (R, A, B) and biases in the models trained with context-based data by using an ensemble model. Our testing results show that the ensemble model constructed performs ~5% better than the best individual context-sensitive models. They also performed better than the models that were directly fine-tuned using mixed imagery data as well as existing deep embeddings, e.g., CLIP [30] and DINO [3]. This work demonstrates that context-based labelling and model training can be effective when an appropriate ensemble approach is used to alleviate the limitation due to sparse sampling.

Fengqi Liang, Huan Zhao 0002, Yuhan Quan, Wei Fang, Chuan Shi 0001

CAD assembly modeling, which refers to using CAD software to design new products from a catalog of existing machine components, is important in the industrial field. The graph neural network (GNN) based recommender system for CAD assembly modeling can help designers make decisions and speed up the design process by recommending the next required component based on the existing components in CAD software. These components can be represented as a graph naturally. However, present recommender systems for CAD assembly modeling adopt fixed GNN architectures, which may be sub-optimal for different manufacturers with different data distribution. Therefore, to customize a well-suited recommender system for different manufacturers, we propose a novel neural architecture search (NAS) framework, dubbed CusGNN, which can design data-specific GNN automatically. Specifically, we design a search space from three dimensions (i.e., aggregation, fusion, and readout functions), which contains a wide variety of GNN architectures. Then, we develop an effective differentiable search algorithm to search high-performing GNN from the search space. Experimental results show that the customized GNNs achieve 1.5-5.1% higher top-10 accuracy compared to previous manual designed methods, demonstrating the superiority of the proposed approach. Code and data are available at https://github.com/BUPT-GAMMA/CusGNN.

Zhixun Li, Yushun Dong, Qiang Liu 0006, Jeffrey Xu Yu

Driven by the powerful representation ability of Graph Neural Networks (GNNs), plentiful GNN models have been widely deployed in many real-world applications. Nevertheless, due to distribution disparities between different demographic groups, fairness in high-stake decision-making systems is receiving increasing attention. Although lots of recent works devoted to improving the fairness of GNNs and achieved considerable success, they all require significant architectural changes or additional loss functions requiring more hyper-parameter tuning. Surprisingly, we find that simple re-balancing methods can easily match or surpass existing fair GNN methods. We claim that the imbalance across different demographic groups is a significant source of unfairness, resulting in imbalanced contributions from each group to the parameters updating. However, these simple re-balancing methods have their own shortcomings during training. In this paper, we propose FairGB, Fair Graph Neural Network via re-Balancing, which mitigates the unfairness of GNNs by group balancing. Technically, FairGB consists of two modules: counterfactual node mixup and contribution alignment loss. Firstly, we select counterfactual pairs across inter-domain and inter-class, and interpolate the ego-networks to generate new samples. Guided by analysis, we can reveal the debiasing mechanism of our model by the causal view and prove that our strategy can make sensitive attributes statistically independent from target labels. Secondly, we reweigh the contribution of each group according to gradients. By combining these two modules, they can mutually promote each other. Experimental results on benchmark datasets show that our method can achieve state-of-the-art results concerning both utility and fairness metrics. Code is available at https://github.com/ZhixunLEE/FairGB.

Yuhan Li 0001, Peisong Wang, Zhixun Li, Jeffrey Xu Yu, Jia Li 0009

With the development of foundation models such as large language models, zero-shot transfer learning has become increasingly significant. This is highlighted by the generative capabilities of NLP models like GPT-4, and the retrieval-based approaches of CV models like CLIP, both of which effectively bridge the gap between seen and unseen data. In the realm of graph learning, the continuous emergence of new graphs and the challenges of human labeling also amplify the necessity for zero-shot transfer learning, driving the exploration of approaches that can generalize across diverse graph data without necessitating dataset-specific and label-specific fine-tuning. In this study, we extend such paradigms to Zero-shot transferability in Graphs by introducing ZeroG, a new framework tailored to enable cross-dataset generalization. Addressing the inherent challenges such as feature misalignment, mismatched label spaces, and negative transfer, we leverage a language model to encode both node attributes and class semantics, ensuring consistent feature dimensions across datasets. We also propose a prompt-based subgraph sampling module that enriches the semantic information and structure information of extracted subgraphs using prompting nodes and neighborhood aggregation, respectively. We further adopt a lightweight fine-tuning strategy that reduces the risk of overfitting and maintains the zero-shot learning efficacy of the language model. The results underscore the effectiveness of our model in achieving significant cross-dataset zero-shot transferability, opening pathways for the development of graph foundation models.

Yuekang Li, Yidan Mao, Yifei Yang 0001, Dongmian Zou

Hyperbolic neural networks (HNNs) are emerging as a promising tool for representing data embedded in non-Euclidean geometries, yet their adoption has been hindered by challenges related to stability and robustness. In this work, we conduct a rigorous Lipschitz analysis for HNNs and propose using Lipschitz regularization as a novel strategy to enhance their robustness. Our comprehensive investigation spans both the Poincaré ball model and the hyperboloid model, establishing Lipschitz bounds for HNN layers. Importantly, our analysis provides detailed insights into the behavior of the Lipschitz bounds as they relate to feature norms, particularly distinguishing between scenarios where features have unit norms and those with large norms. Further, we study regularization using the derived Lipschitz bounds. Our empirical validations demonstrate consistent improvements in HNN robustness against noisy perturbations.

Yicong Li 0001, Yu Yang 0012, Jiannong Cao 0001, Shuaiqi Liu 0002, Haoran Tang, Guandong Xu

Recent studies successfully learned static graph embeddings that are structurally fair by preventing the effectiveness disparity of high- and low-degree vertex groups in downstream graph mining tasks. However, achieving structure fairness in dynamic graph embedding remains an open problem. Neglecting degree changes in dynamic graphs will significantly impair embedding effectiveness without notably improving structure fairness. This is because the embedding performance of high-degree and low-to-high-degree vertices will significantly drop close to the generally poorer embedding performance of most slightly changed vertices in the long-tail part of the power-law distribution. We first identify biased structural evolutions in a dynamic graph based on the evolving trend of vertex degree and then propose FairDGE, the first structurally Fair Dynamic Graph Embedding algorithm. FairDGE learns biased structural evolutions by jointly embedding the connection changes among vertices and the long-short-term evolutionary trend of vertex degrees. Furthermore, a novel dual debiasing approach is devised to encode fair embeddings contrastively, customizing debiasing strategies for different biased structural evolutions. This innovative debiasing strategy breaks the effectiveness bottleneck of embeddings without notable fairness loss. Extensive experiments demonstrate that FairDGE achieves simultaneous improvement in the effectiveness and fairness of embeddings.

Yan Li 0165, Yundu Huang, Wuyang Mao, Furong Ye, Xiang He 0005, Zhonglin Zu, Shaowei Cai 0001

Contemporary systems of Guaranteed Delivery (GD) advertising work with two different stages, namely, the offline selling stage and the online serving stage. The former deals with contract allocation, and the latter fulfills the impression allocation of signed contracts. Existing work usually handles these two stages separately. For example, contracts are formulated offline without concerning practical situations in the online serving stage. Therefore, we address in this paper a bi-objective contract allocation for GD advertising, which maximizes the impressions, i.e., Ad resource assignments, allocated for the new incoming advertising orders, and at the same time, controls the balance in the inventories. Since the proposed problem is high dimensional and heavily constrained, we design an efficient local search that focuses on the two objectives alternatively. The experimental results indicate that our algorithm outperforms multi-objective evolutionary algorithms and Gurobi, the former of which is commonly applied for multi-objective optimization and the latter of which is a well-known competitive commercial tool.

Xujia Li, Jingshu Peng, Lei Chen 0002

In today's fast-paced world, advertisers are increasingly demanding real-time and accurate personalized ad delivery based on dynamic preference modeling, which emphasizes the temporality existing in both user preference and product characteristics. Meanwhile, with the development of graph neural networks (GNNs), E-commerce knowledge graphs (KG) with rich semantic relatedness are invoked to improve accuracy and provide appropriate explanations to encourage advertisers' willingness to invest in ad expenses. However, it is still challenging for existing methods to comprehensively consider both time-series interactions and graph-structured knowledge triples in a unified model, i.e., the case in knowledge-aware dynamic advertising. The interaction graph between users and products changes rapidly over time, while the knowledge in KG remains relatively stable. This results in an uneven distribution of temporal and semantic information, causing existing GNNs to fail in this scenario. In this work, we quantitatively define the above phenomenon as temporal unevenness and introduce the Incremental Leveling Network (InLN) with three novel techniques: the periodic-focusing window for node-level dynamic modeling, the biased temporal walk for subgraph-level dynamic modeling and the incremental leveling mechanism for KG updating. Verified by comprehensive and intensive experiments, InLN outperforms nine baseline models in three tasks by substantial margins, reaching up to a 9.9% improvement and averaging a 5.7% increase.

Xinshu Li 0001, Mingming Gong, Lina Yao 0001

The advancements in disentangled representation learning significantly enhance the accuracy of counterfactual predictions by granting precise control over instrumental variables, confounders, and adjustable variables. An appealing method for achieving the independent separation of these factors is mutual information minimization, a task that presents challenges in numerous machine learning scenarios, especially within high-dimensional spaces. To circumvent this challenge, we propose the Self-Distilled Disentanglement framework, referred to as SD2. Grounded in information theory, it ensures theoretically sound independent disentangled representations without intricate mutual information estimator designs for high-dimensional representations. Our comprehensive experiments, conducted on both synthetic and real-world datasets, confirms the effectiveness of our approach in facilitating counterfactual inference in the presence of both observed and unobserved confounders.

Ruikun Li 0002, Huandong Wang, Jinghua Piao, Qingmin Liao, Yong Li 0008

Learning complex network dynamics is fundamental for understanding, modeling, and controlling real-world complex systems. Though great efforts have been made to predict the future states of nodes on networks, the capability of capturing long-term dynamics remains largely limited. This is because they overlook the fact that long-term dynamics in complex network are predominantly governed by their inherent low-dimensional manifolds, i.e., skeletons. Therefore, we propose the Dynamics-Invariant Skeleton Neural Network (DiskNet), which identifies skeletons of complex networks based on the renormalization group structure in hyperbolic space to preserve both topological and dynamics properties. Specifically, we first condense complex networks with various dynamics into simple skeletons through physics-informed hyperbolic embeddings. Further, we design graph neural ordinary differential equations to capture the condensed dynamics on the skeletons. Finally, we recover the skeleton networks and dynamics to the original ones using a degree-based super-resolution module. Extensive experiments across three representative dynamics as well as five real-world and two synthetic networks demonstrate the superior performances of the proposed DiskNet, which outperforms the state-of-the-art baselines by an average of 10.18% in terms of long-term prediction accuracy. Code for reproduction is available at: https://github.com/tsinghua-fib-lab/DiskNet.

Rongqing Li, Changsheng Li, Yuhang Li 0007, Hanjie Li, Yi Chen 0031, Ye Yuan 0001, Guoren Wang

Trajectory prediction of moving traffic agents is crucial for the safety of autonomous vehicles, whereas previous approaches usually rely on sufficiently long-observed trajectory (e.g., 2 seconds) to predict the future trajectory of the agents. However, in many real-world scenarios, it is not realistic to collect adequate observed locations for moving agents, leading to the collapse of most prediction models. For instance, when a moving car suddenly appears and is very close to an autonomous vehicle because of the obstruction, it is quite necessary for the autonomous vehicle to quickly and accurately predict the future trajectories of the car with limited observed trajectory locations. In light of this, we focus on investigating the task of instantaneous trajectory prediction, i.e., two observed locations are available during inference. To this end, we put forward a general and plug-and-play instantaneous trajectory prediction approach, called ITPNet. Specifically, we propose a backward forecasting mechanism to reversely predict the latent feature representations of unobserved historical trajectories of the agent based on its two observed locations and then leverage them as complementary information for future trajectory prediction. Meanwhile, due to the inevitable existence of noise and redundancy in the predicted latent feature representations, we further devise a Noise Redundancy Reduction Former (NRRFormer) module, which aims to filter out noise and redundancy from unobserved trajectories and integrate the filtered features and observed features into a compact query representation for future trajectory predictions. In essence, ITPNet can be naturally compatible with existing trajectory prediction models, enabling them to gracefully handle the case of instantaneous trajectory prediction. Extensive experiments on the Argoverse and nuScenes datasets demonstrate ITPNet outperforms the baselines by a large margin and shows its efficacy with different trajectory prediction models.

Ran Li, Shimin Di, Lei Chen 0002, Xiaofang Zhou 0001

In this paper, we address the challenges of data augmentation in Multi-Modal Knowledge Graphs (MMKGs), a relatively under-explored area. We propose a novel diffusion-based generative model, the Simple Denoising Probabilistic Latent Diffusion Model (SimDiff). SimDiff is capable of handling different data modalities including the graph topology in a unified manner by the same diffusion model in the latent space. It enhances the utilization of multi-modal data and encourage the multi-modal fusion and reduces the dependency on limited training data. We validate our method in downstream Entity Alignment (EA) tasks in MMKGs, demonstrating that even when using only half of the seed entities in training, our methods can still achieve superior performance. This work contributes to the field by providing a new data generation or augmentation method for MMKGs, potentially paving the way for more effective use of MMKGs in various applications. Code is made available at https://github.com/ranlislz/SimDiff.

Qingyao Li, Wei Xia 0001, Li'ang Yin, Jiarui Jin, Yong Yu 0001

Educational recommendation seeks to suggest knowledge concepts that match a learner's ability, thus facilitating a personalized learning experience. In recent years, reinforcement learning (RL) methods have achieved considerable results by taking the encoding of the learner's exercise log as the state and employing an RL-based agent to make suitable recommendations. However, these approaches suffer from handling the diverse and dynamic learner's knowledge states. In this paper, we introduce the privileged feature distillation technique and propose the P rivileged K nowledge S tate D istillation (PKSD ) framework, allowing the RL agent to leverage the "actual'' knowledge state as privileged information in the state encoding to help tailor recommendations to meet individual needs. Concretely, our PKSD takes the privileged knowledge states together with the representations of the exercise log for the state representations during training. And through distillation, we transfer the ability to adapt to learners to aknowledge state adapter. During inference, theknowledge state adapter would serve as the estimated privileged knowledge states instead of the real one since it is not accessible. Considering that there are strong connections among the knowledge concepts in education, we further propose to collaborate the graph structure learning for concepts into our PKSD framework. This new approach is termed GEPKSD (Graph-Enhanced PKSD). As our method is model-agnostic, we evaluate PKSD and GEPKSD by integrating them with five different RL bases on four public simulators, respectively. Our results verify that PKSD can consistently improve the recommendation performance with various RL methods, and our GEPKSD could further enhance the effectiveness of PKSD in all the simulations.

Mei Li, Xiaoguang Liu 0001, Hua Ji, Shuangjia Zheng

Inductive relation reasoning in knowledge graphs aims at predicting missing triplets involving unseen entities and/or unseen relations. While subgraph-based methods that reason about the local structure surrounding a candidate triplet have shown promise, they often fall short in accurately modeling the causal dependence between a triplet's subgraph and its ground-truth label. This limitation typically results in a susceptibility to spurious correlations caused by confounders, adversely affecting generalization capabilities. Herein, we introduce a novel front-door adjustment-based approach designed to learn the causal relationship between subgraphs and their ground-truth labels, specifically for inductive relation prediction. We conceptualize the semantic information of subgraphs as a mediator and employ a graph data augmentation mechanism to create augmented subgraphs. Furthermore, we integrate a fusion module and a decoder within the front-door adjustment framework, enabling the estimation of the mediator's combination with augmented subgraphs. We also introduce the reparameterization trick in the fusion model to enhance model robustness. Extensive experiments on widely recognized benchmark datasets demonstrate the proposed method's superiority in inductive relation prediction, particularly for tasks involving unseen entities and unseen relations. Additionally, the subgraphs reconstructed by our decoder offer valuable insights into the model's decision-making process, enhancing transparency and interpretability.

Jing Li 0026, Quanxue Gao, Qianqian Wang 0001, Cheng Deng 0002, De-Yan Xie

Multi-view clustering method based on anchor graph has been widely concerned due to its high efficiency and effectiveness. In order to avoid post-processing, most of the existing anchor graph-based methods learn bipartite graphs with connected components. However, such methods have high requirements on parameters, and in some cases it may not be possible to obtain bipartite graphs with clear connected components. To end this, we propose a label learning method based on tensor projection (LLMTP). Specifically, we project anchor graph into the label space through an orthogonal projection matrix to obtain cluster labels directly. Considering that the spatial structure information of multi-view data may be ignored to a certain extent when projected in different views separately, we extend the matrix projection transformation to tensor projection, so that the spatial structure information between views can be fully utilized. In addition, we introduce the tensor Schatten p-norm regularization to make the clustering label matrices of different views as consistent as possible. Extensive experiments have proved the effectiveness of the proposed method.