论文检索

Shuhan Zhong, Weipeng Zhuo, Sizhe Song, Guanyao Li, Zhongyi Yu, S.-H. Gary Chan

Irregular multivariate time series (IMTS) is characterized by the lack of synchronized observations across its different channels. In this paper, we point out that this channel-wise asynchrony can lead to poor channel-wise modeling of existing deep learning methods. To overcome this limitation, we propose MTM, a multi-scale token mixing transformer for the classification of IMTS. We find that the channel-wise asynchrony can be alleviated by down-sampling the time series to coarser timescales, and propose to incorporate a masked concat pooling in MTM that gradually down-samples IMTS to enhance the channel-wise attention modules. Meanwhile, we propose a novel channel-wise token mixing mechanism which proactively chooses important tokens from one channel and mixes them with other channels, to further boost the channel-wise learning of our model. Through extensive experiments on real-world datasets and comparison with state-of-the-art methods, we demonstrate that MTM consistently achieves the best performance on all the benchmarks, with improvements of up to 3.8% in AUPRC for classification.

Luying Zhong, Xuan Lai, Junjie Zhang 0010, Zhiqin Huang, Zheyi Chen

The emerging Federated Graph Learning (FGL) offers promising collaborative training on distributed graph data. However, malicious actors may contaminate data streams by falsifying node relationships on clients or conduct adversarial attacks on edge servers, causing degraded inference and privacy leakage. Although some studies focus on privacy-protection FGL, they do not consider robustness and membership privacy amidst data pollution and adversarial attacks. Moreover, classic FGL commonly adopts FedAvg but neglects the impact of uneven information flow from distinct subtopologies. To address these important challenges, we propose GuardFGL, a novel similarity-driven FGL that extracts minimal-sufficient information from polluted data to maintain strong adversarial robustness and protect membership privacy. First, we incorporate structural-aware and feature-selection learning to explore target-relevant edges and features, avoiding privacy leakage from raw data. Next, we design an original Federated Graph Information Bottleneck (FGIB) principle to supervise extracting well-compressed information, mitigating the interference of polluted data streams. Finally, we develop a similarity-driven federated aggregation with auxiliary local information to alleviate the impact of uneven information flow. Using the real-world testbed and benchmark graph datasets, extensive experiments demonstrate that GuardFGL can achieve superior robust prediction and better protect membership privacy than state-of-the-art methods under adversarial attacks.

Ziyu Zheng, Yaming Yang 0002, Ziyu Guan, Wei Zhao 0019, Weigang Lu 0001

Real-world networks usually have a property of node heterophily, that is, the connected nodes usually have different features or different labels. This heterophily issue has been extensively studied in homogeneous graphs but remains under-explored in heterogeneous graphs, where there are multiple types of nodes and edges. Capturing node heterophily in heterogeneous graphs is very challenging since both node/edge heterogeneity and node heterophily should be carefully taken into consideration. Existing methods typically convert heterogeneous graphs into homogeneous ones to learn node heterophily, which will inevitably lose the potential heterophily conveyed by heterogeneous relations. To bridge this gap, we propose Relation-Aware Separation of Homophily and Heterophily (RASH), a novel contrastive learning framework that explicitly models high-order semantics of heterogeneous interactions and adaptively separates homophilic and heterophilic patterns. Particularly, RASH introduces dual heterogeneous hypergraphs to encode multi-relational bipartite subgraphs and dynamically constructs homophilic graphs and heterophilic graphs based on relation importance. A multi-relation contrastive loss is designed to align heterogeneous and homophilic/heterophilic views by maximizing mutual information. In this way, RASH simultaneously resolves the challenges of heterogeneity and heterophily in heterogeneous graphs. Extensive experiments on benchmark datasets demonstrate the effectiveness of RASH across various downstream tasks. The code is available at: https://github.com/zhengziyu77/RASH.

Ziyu Zheng, Yaming Yang 0002, Ziyu Guan, Wei Zhao 0019, Weigang Lu 0001

Masked Graph Auto-Encoder, a powerful graph self-supervised training paradigm, has recently shown superior performance in graph representation learning. Existing works typically rely on node contextual information to recover the masked information. However, they fail to generalize well to heterophilic graphs where connected nodes may be not similar, because they focus only on capturing the neighborhood information and ignoring the discrepancy information between different nodes, resulting in indistinguishable node representations. In this paper, to address this issue, we propose a Discrepancy-Aware Graph Mask Auto-Encoder (DGMAE). It obtains more distinguishable node representations by reconstructing the discrepancy information of neighboring nodes during the masking process. We conduct extensive experiments on 17 widely-used benchmark datasets. The results show that our DGMAE can effectively preserve the discrepancies of nodes in low-dimensional space. Moreover, DGMAE significantly outperforms state-of-the-art graph self-supervised learning methods on three graph analytic including tasks node classification, node clustering, and graph classification, demonstrating its remarkable superiority. The code of DGMAE is available at https://github.com/zhengziyu77/DGMAE.

Liangwei Nathan Zheng, Chang George Dong, Wei Emma Zhang, Lin Yue, Miao Xu 0001, Olaf Maennel, Weitong Chen 0001

Large Language Models (LLMs) have demonstrated impressive performance in time series analysis and seems to understand the time temporal relationship well than traditional transformer-based approaches. However, since LLMs are not designed for time series tasks, simpler models-like linear regressions can often achieve comparable performance with far less complexity. In this study, we perform extensive experiments to assess the effectiveness of applying LLMs to key time series tasks, including forecasting, classification, imputation, and anomaly detection. We compare the performance of LLMs against simpler baseline models, such as single-layer linear models and randomly initialized LLMs. Our results reveal that LLMs offer minimal advantages for these core time series tasks and may even distort the temporal structure of the data. In contrast, simpler models consistently outperform LLMs while requiring far fewer parameters. Furthermore, we analyze existing reprogramming techniques and show, through data manifold analysis, that these methods fail to effectively align time series data with language and display ''pseudo-alignment'' behavior in embedding space. Our findings suggest that the performance of LLM-based methods in time series tasks arises from the intrinsic characteristics and structure of time series data, rather than any meaningful alignment with the language model architecture. We release the code for experiments here: https://github.com/IcurasLW/Official-Repository_Understanding_LLM_for_Time_Series_Analysis.git

Jingyuan Zheng, Xin Zhang 0079, Zhilin Qi, Ruiang Qiu, Dongjing Wang, Haiping Zhang 0001, Dongjin Yu

Extreme precipitation, as a core causative factor of meteorological disasters, poses significant challenges for accurate short-term forecasting due to the chaotic nature of precipitation systems and their multi-scale spatio-temporal evolution. Traditional numerical models are notably affected by error accumulation, while existing deep learning models still face dual limitations in physical fidelity and multi-scale feature extraction. To Address these issues, we propose an innovative Multi-scale Physics-informed Transformer with spatio-temporal feature adapter for extreme precipitation nowcasting, termed MPFormer. Our framework comprises two core components: the deterministic Evolution Network and the stochastic Generative Network. The Evolution Network integrates a novel Scale-Aware Temporal Residual Modulation Transformer (STRMT) encoder that captures multi-scale storm dynamics through residual temporal attention. The Generative Network introduces spatio-temporal adapters as lightweight transfer modules for probabilistic modeling. We develop a Multi-scale Physics-informed Loss with three innovations: 1) dynamic weight scheduling for feature fusion, 2) physical constraints preserving storm evolution patterns, and 3) entropy-based uncertainty calibration. Experiments based on MRMS radar data from North America demonstrate that the model can generate high-resolution forecasts (2km grid) with a 3-hour lead time over an area of 2048×2048 square kilometers. Compared to the state-of-the-art technologies, the proposed framework shows significant effectiveness and superiority in metrics such as CSIN, offering a new paradigm that combines physical interpretability with engineering practicality for extreme weather warnings and disaster prevention in smart cities.

Ziqi Zhao, Zhaochun Ren, Liu Yang 0025, Yunsen Liang, Fajie Yuan, Pengjie Ren, Zhumin Chen, Jun Ma 0001, Xin Xin 0003

Offline reinforcement learning (RL) aims to learn policies without online explorations. To enlarge the training data, model-based offline RL learns a dynamics model which is utilized as a virtual environment to generate simulation data and enhance policy learning. However, existing data augmentation methods for offline RL suffer from (i) trivial improvement from short-horizon simulation; and (ii) the lack of evaluation and correction for generated data, leading to low-qualified augmentation. In this paper, we propose offline trajectory optimization for offline reinforcement learning (OTTO). The key motivation is to conduct long-horizon simulation and then utilize model uncertainty to evaluate and correct the augmented data. Specifically, we propose an ensemble of Transformers, a.k.a. World Transformers, to predict environment state dynamics and the reward function. Three strategies are proposed to use World Transformers to generate long-horizon trajectory simulation by perturbing the actions in the offline data. Then, an uncertainty-based World Evaluator is introduced to firstly evaluate the confidence of the generated trajectories and then perform the correction for low-confidence data. Finally, we jointly use the original data with the corrected augmentation data to train an offline RL algorithm. OTTO serves as a plug-in module and can be integrated with existing model-free offline RL methods. Experiments on various benchmarks show that OTTO can effectively improve the performance of representative offline RL algorithms, including in complex environments with sparse rewards like AntMaze. Codes are available at https://github.com/ZiqiZhao1/OTTO

Ziliang Zhao 0001, Changle Qu, Zhicheng Dou, Haonan Chen 0005, Jiajie Jin

Search clarification is a critical user interface for open-domain conversational Web search, where generating high-quality facets for ambiguous or multi-facet queries significantly guides disambiguation and enhances the user's interaction experience. Recently, in-context learning with Large Language Models (LLMs) has emerged as a promising approach for facet generation by leveraging static or similarity-based demonstrations as prompts. However, existing methods predominantly rely on query similarity, failing to account for the multi-dimensional nature of query intents. This limitation can lead LLMs to generate incorrect or suboptimal facets misaligned with user needs. To address this challenge, we propose an intent-covering framework that improves clarification facet generation by selecting demonstrations that comprehensively cover the diverse intents underlying a given query. Specifically, we first train a generative model with beam search to predict potential intents and construct an intent-document graph to capture their semantic relationships. We then introduce a heuristic greedy algorithm that optimizes demonstration selection by maximizing intent coverage. Furthermore, since the order of demonstrations significantly affects generation quality, we develop a re-ranking model to optimize their sequence for better contextual alignment. Experiments demonstrate the superiority of our approach over strong baselines in various lexical and semantic evaluation metrics. Additionally, we conduct an in-depth analysis of how the number, order, and contextual relevance of demonstrations influence generation performance.

Zhuowei Zhao, Junhao Gan, Boyu Ruan, Zhifeng Bao, Jianzhong Qi 0001, Sibo Wang 0001

We study structural clustering on graphs in dynamic scenarios, where graphs can be updated by arbitrary insertions or deletions of edges/vertices. Our goal is to efficiently compute structural clustering results under three conditions: 1) for any clustering parameters ε and μ provided on the fly, 2) for arbitrary graph update patterns, and 3) for all typical similarity measurements. To achieve this, we propose an algorithm named VD-STAR that is much simpler yet more efficient than state of the art. With a theoretical guarantee on clustering result's quality, VD-STAR can produce clustering results with up to 99.9% accuracy. Moreover, VD-STAR is easy to implement as it just needs to maintain sorted linked lists and hash tables, making it highly deployable in practice. Most importantly, VD-STAR improves the expected per-update time bound from state-of-the-art O(log2 n), which relies on specific assumption on update pattern, to O(log n) amortized in expectation without any assumption on update pattern. We further design two variants of VD-STAR to enhance its empirical performance. Experimental results show that our algorithms consistently outperform state-of-the-art competitors by up to 9,315 times in update time across nine real datasets, while maintaining similar update time and memory usage.

Zhiyuan Zhao 0002, Haoxin Liu 0001, Alexander Rodríguez, B. Aditya Prakash

Time-series forecasting is a critical challenge in various domains and has witnessed substantial progress in recent years. Many real-life scenarios, such as public health, economics, and social applications, involve feedback loops where predictive models can trigger actions that influence the outcome they aim to predict, subsequently altering the target variable's distribution. This phenomenon, known as performativity, introduces the potential for 'self-negating' or 'self-fulfilling' predictions. Despite extensive studies on performativity in classification problems across domains, this phenomenon remains largely unexplored in the context of time-series forecasting from a machine-learning perspective. In this paper, we formalize Performative Time-Series Forecasting (PeTS), addressing the challenge for predictions when performativity-induced distribution shifts are possible. We propose a novel solution to PeTS, Feature Performative-Shifting (FPS), which leverages the concept of delayed response to anticipate distribution shifts and subsequently predicts targets. We provide theoretical insights suggesting that FPS potentially leads to reduced generalization error. Extensive experiment results demonstrate that FPS consistently outperforms conventional time-series forecasting and concept drift methods, highlighting its efficacy in handling performativity-induced challenges.

Yumeng Zhao, Hongxiang Lin, Shuo Wen, Junjie Shen, Bei Hua

Recommender Systems (RS) play a critical role in enhancing user experiences across online platforms by modeling user-item interactions as bipartite graphs. Predicting signed links in such graphs remains challenging due to the sparsity and complexity of sign distributions and the limitations of traditional methods like matrix factorization and Graph Convolutional Networks (GCNs), which often fail to capture the intricate local topological and sign-based patterns essential for accurate predictions. To address these challenges, we propose SMA-GNN, a framework specifically designed for signed link prediction in bipartite graphs. SMA-GNN combines Local Subgraph Extraction, Two-Anchor Distance Labeling (TADL), and a Symbol-aware Multi-head Attention Mechanism to enhance predictive capability and interpretability. By extracting a closed local subgraph around the target link, our method captures relevant topological and sign contexts. TADL refines this by assigning unique structural labels to nodes based on their proximity to anchor nodes, encapsulating roles and relationships. The symbol-aware attention mechanism integrates edge sign information into the message-passing process, generating highly discriminative subgraph embeddings. Experiments on benchmark datasets show that SMA-GNN outperforms global embedding methods in prediction accuracy and provides deeper insights into user-item interactions, enabling more precise and personalized recommendations. Our code is avilable at https://github.com/xiaohuzidefeijian/SMAGNN/tree/master

Kai Zhao 0009, Zhihao Zhuang, Chenjuan Guo, Hao Miao 0001, Christian S. Jensen, Yunyao Cheng 0001, Bin Yang 0002

We study the problem of time series anomaly prediction, which is relevant to a range of real-world applications. Existing anomaly prediction methods rely on labeled training data for achieving acceptable accuracy. However, such data may be difficult to obtain; and in real-time deployments, anomalies can occur that were not seen in labeled data, thus making them difficult to predict. We provide a theoretical analysis and propose an Importance-based Generative Contrastive Learning method (IGCL) for unsupervised anomaly prediction. IGCL employs a controlled diffusion module to produce anomaly precursor patterns. Next, ICGL learns contextual representations to extract temporal dependencies from pairs of normal time series and anomaly precursors. IGCL is then able to predict anomalies by identifying anomaly precursors that will evolve into future anomalies. To address challenges caused by potentially complex precursor combinations involving multiple variables, we propose a memory bank with importance scores that stores representative samples adaptively and generates more complex anomaly precursors. Extensive experiments on nine benchmark datasets offer evidence that the proposed method is able to outperform state-of-the-art baselines.

Chuang Zhao 0002, Hui Tang, Hongke Zhao, Xiaomeng Li 0001

Advanced healthcare predictions offer significant improvements in patient outcomes by leveraging predictive analytics. Existing works primarily utilize various views of Electronic Health Record (EHR) data, such as diagnoses, lab tests, or clinical notes, for model training. These methods typically assume the availability of complete EHR views and that the designed model could fully leverage the potential of each view. However, in practice, random missing views and view laziness present two significant challenges that hinder further improvements in multi-view utilization. To address these challenges, we introduce Diffmv, an innovative diffusion-based generative framework designed to advance the exploitation of multiple views of EHR data. Specifically, to address random missing views, we integrate various views of EHR data into a unified diffusion-denoising framework, enriched with diverse contextual conditions to facilitate progressive alignment and view transformation. To mitigate view laziness, we propose a novel reweighting strategy that assesses the relative advantages of each view, promoting a balanced utilization of various data views within the model. Our proposed strategy achieves superior performance across multiple health prediction tasks derived from three popular datasets, including multi-view and multi-modality scenarios.

Zhongjin Zhang, Yu Liang, Cong Fu 0001, Yuxuan Zhu 0001, Kun Wang 0024, Yabo Ni, Anxiang Zeng, Jiazhi Xia

Embedding-based collaborative filtering, often coupled with nearest neighbor search, is widely deployed in large-scale recommender systems for personalized content selection. Modern systems leverage multiple implicit feedback signals (e.g., clicks, add to cart, purchases) to model user preferences comprehensively. However, prevailing approaches adopt a feedback-wise modeling paradigm, which (1) fails to capture the structured progression of user engagement entailed among different feedback and (2) embeds feedback-specific information into disjoint spaces, making representations incommensurable, increasing system complexity, and leading to suboptimal retrieval performance. A promising alternative is Ordinal Logistic Regression (OLR), which explicitly models discrete ordered relations. However, existing OLR-based recommendation models mainly focus on explicit feedback (e.g., movie ratings) and struggle with implicit, correlated feedback, where ordering is vague and non-linear. Moreover, standard OLR lacks flexibility in handling feedback-dependent covariates, resulting in suboptimal performance in real-world systems. To address these limitations, we propose Generalized Neural Ordinal Logistic Regression (GNOLR), which encodes multiple feature-feedback dependencies into a unified, structured embedding space and enforces feedback-specific dependency learning through a nested optimization framework. Thus, GNOLR enhances predictive accuracy, captures the progression of user engagement, and simplifies the retrieval process. We establish a theoretical comparison with existing paradigms, demonstrating how GNOLR avoids disjoint spaces while maintaining effectiveness. Extensive experiments on ten real-world datasets show that GNOLR significantly outperforms state-of-the-art methods in efficiency and adaptability.

Zhaoyang Zhang, Ziqi Chen 0002, Qiao Liu 0008, Jinhan Xie, Hongtu Zhu

In this paper, we propose a novel framework, the Sampling-guided Heterogeneous Graph Neural Network (HT-GNN), to effectively tackle the challenge of missing data imputation in longitudinal studies. Unlike traditional methods, which often require extensive preprocessing to handle irregular or inconsistent missing data, our approach accommodates arbitrary missing data patterns while maintaining computational efficiency. HT-GNN models both observations and covariates as distinct node types, connecting observation nodes at successive time points through subject-specific longitudinal subnetworks, while covariate-observation interactions are represented by attributed edges within bipartite graphs. By leveraging subject-wise mini-batch sampling and a multi-layer temporal smoothing mechanism, HT-GNN efficiently scales to large datasets, while effectively learning node representations and imputing missing data. Extensive experiments on both synthetic and real-world datasets, including the Alzheimer's Disease Neuroimaging Initiative (DNI) dataset, demonstrate that HT-GNN significantly outperforms existing imputation methods, even with high missing data rates (e.g., 80%). The empirical results highlight HT-GNN's robust imputation capabilities and superior performance, particularly in the context of complex, large-scale longitudinal data.

Yiqing Zhang, Xiaozhong Liu 0001, Fabricio Murai

Many existing models for clinical trial outcome prediction are optimized using task-specific loss functions on trial phase-specific data. While this scheme may boost prediction for common diseases and drugs, it can hinder the learning of generalizable representations, leading to more false positives/negatives. To address this limitation, we introduce CLaDMoP, a new pre-training approach for clinical trial outcome prediction, alongside the Successful Clinical Trials dataset (SCT), specifically designed for this task. CLaDMoP leverages a Large Language Model-to encode trials' eligibility criteria-linked to a lightweight Drug-Molecule branch through a novel multi-level fusion technique. To efficiently fuse long embeddings across levels, we incorporate a grouping block, drastically reducing computational overhead. CLaDMoP avoids reliance on task-specific objectives by pre-training on a ''pair matching'' proxy task. Compared to established zero-shot and few-shot baselines, our method significantly improves both PR-AUC and ROC-AUC, especially for phase I and phase II trials. We further evaluate and perform ablation on CLaDMoP after Parameter-Efficient Fine-Tuning, comparing it to state-of-the-art supervised baselines, including MEXA-CTP, on the Trial Outcome Prediction (TOP) benchmark. CLaDMoP achieves up to 10.5% improvement in PR-AUC and 3.6% in ROC-AUC, while attaining comparable F1 score to MEXA-CTP, highlighting its potential for clinical trial outcome prediction. Code and SCT dataset can be downloaded from https://github.com/murai-lab/CLaDMoP.

Yingyi Zhang 0001, Pengyue Jia, Xianneng Li, Derong Xu, Maolin Wang 0001, Yichao Wang 0002, Zhaocheng Du, Huifeng Guo, Yong Liu 0020, Ruiming Tang 等

Cloud-device collaboration leverages on-cloud Large Language Models (LLMs) for handling public user queries and on-device Small Language Models (SLMs) for processing private user data, collectively forming a powerful and privacy-preserving solution. However, existing approaches often fail to fully leverage the scalable problem-solving capabilities of on-cloud LLMs while underutilizing the advantage of on-device SLMs in accessing and processing personalized data. This leads to two interconnected issues: 1) Limited utilization of the problem-solving capabilities of on-cloud LLMs, which fail to align with personalized user-task needs, and 2) Inadequate integration of user data into on-device SLM responses, resulting in mismatches in contextual user information. In this paper, we propose a Leader-Subordinate Retrieval framework for Privacy-preserving cloud-device collaboration (LSRP), a novel solution that bridges these gaps by: 1) enhancing on-cloud LLM guidance to on-device SLM through a dynamic selection of task-specific leader strategies named as user-to-user retrieval-augmented generation (U-U-RAG), and 2) integrating the data advantages of on-device SLMs through small model feedback Direct Preference Optimization (SMFB-DPO) for aligning the on-cloud LLM with the on-device SLM. Experiments on two datasets demonstrate that LSRP consistently outperforms state-of-the-art baselines, significantly improving question-answer relevance and personalization, while preserving user privacy through efficient on-device retrieval. Our code is available at: https://github.com/Applied-Machine-Learning-Lab/LSRP.

Yansen Zhang, Xiaokun Zhang 0001, Ziqiang Cui, Chen Ma 0001

Recommender systems often suffer from noisy interactions like accidental clicks or popularity bias. Existing denoising methods typically identify users' intent in their interactions, and filter out noisy interactions that deviate from the assumed intent. However, they ignore that interactions deemed noisy could still aid model training, while some ''clean'' interactions offer little learning value. To bridge this gap, we propose Shapley Value-driven Valuation (SVV), a framework that evaluates interactions based on their objective impact on model training rather than subjective intent assumptions. In SVV, a real-time Shapley value estimation method is devised to quantify each interaction's value based on its contribution to reducing training loss. Afterward, SVV highlights the interactions with high values while downplaying low ones to achieve effective data pruning for recommender systems. In addition, we develop a simulated noise protocol to examine the performance of various denoising approaches systematically. Experiments on four real-world datasets show that SVV outperforms existing denoising methods in both accuracy and robustness. Further analysis also demonstrates that our SVV can preserve training-critical interactions and offer interpretable noise assessment. This work shifts denoising from heuristic filtering to principled, model-driven interaction valuation.

Yang Zhang 0058, Wenbo Yang, Jun Wang 0089, Qiang Ma 0001, Jie Xiong 0008

该来源暂未提供摘要。

Yan Zhang 0111, Xiaoye Miao, Bin Li, Yangyang Wu, Yongheng Shang

Federated data selection strategically chooses a group of high-quality samples to train a global model, and it is promising to optimize the convergence and resource overhead of federated learning (FL). However, existing studies either fail to account for the dynamic importance of training samples or rely on external unbiased validation datasets. These shortcomings can compromise FL model performance, potentially complicating their application in real-world scenarios. In this paper, we propose a novel proxy-validated importance-aware federated sample selection framework, termed FedSelect. It employs a novel meta learning approach with a proxy validation dataset to select the most positively important clients and their most important samples, thereby accelerating the training process and optimizing FL model performance. To eliminate the dependency on external unbiased data, we present a momentum-based meta-margin function to discover influential samples as the proxy validation dataset, providing an adaptive reward for sample selection. Furthermore, we also develop an online meta model update strategy to guarantee the efficiency of FedSelect. Comprehensive experiments on four benchmark datasets demonstrate that FedSelect is superior in both effectiveness and efficiency, while maintaining strong scalability across diverse scenarios. The source code can be accessed at: https://github.com/nameyzhang/FedSelect.