Mobile traffic forecasting allows operators to anticipate network dynamics and performance in advance, offering substantial potential for enhancing service quality and improving user experience. It involves multiple tasks, including long-term prediction, short-term prediction, and generation tasks that do not rely on historical data. By leveraging the different types of mobile network data generated from these tasks, operators can perform a variety of network optimizations and planning activities, such as base station (BS) deployment, resource allocation, energy optimization, etc. However, existing models are often designed for specific tasks and trained with specialized data, and there is a lack of universal models for traffic forecasting across different urban environments. In this paper, we propose a Universal model for Mobile traffic forecasting (UoMo), aiming to handle diverse forecasting tasks of short/long-term predictions and distribution generation across multiple cities to support network planning and optimization. UoMo combines diffusion models and transformers, where various spatio-temporal masks are proposed to enable UoMo to learn intrinsic features of different tasks, and a contrastive learning strategy is developed to capture the correlations between mobile traffic and urban contexts, thereby improving its transfer learning capability. Extensive evaluations on 9 real-world datasets demonstrate that UoMo outperforms current models in various forecasting tasks and zero/few-shot learning. It shows an average accuracy improvement of 27.85%, 18.57%, and 15.6% in long-term prediction, short-term prediction, and generation tasks, respectively, showcasing its strong forecasting capability. We deploy UoMo on China Mobile's JiuTian platform, leveraging the predicted mobile data to optimize live networks. This optimization includes BS deployment, resulting in a 25.3% increase in served users, and BS sleep control, which reduces equipment depreciation by 40.7%. The source code is available online: https://github.com/tsinghua-fib-lab/UoMo.
论文检索
输入标题、作者或关键词，从 844 篇学术成果中精准定位
With the rapid integration of Machine Learning (ML) in business applications and processes, it is crucial to ensure the quality, reliability and reproducibility of such systems. We suggest a methodical approach towards ML system quality assessment and introduce a structured Maturity framework for governance of ML. We emphasize the importance of quality in ML and the need for rigorous assessment, driven by issues in ML governance and gaps in existing frameworks. Our primary contribution is a comprehensive open-sourced quality assessment method, validated with empirical evidence, accompanied by a systematic maturity framework tailored to ML systems. Drawing from applied experience at Booking.com, we discuss challenges and lessons learned during large-scale adoption within organizations. The study presents empirical findings, highlighting quality improvement trends and showcasing business outcomes. The maturity framework for ML systems, aims to become a valuable resource to reshape industry standards and enable a structural approach to improve ML maturity in any organization.
Recommendation systems are effective tools for information filtering and discovery. These systems are widely applied across various consumer sectors and hold significant potential for applications in professional domains to enhance work efficiency. However, supporting decision-making in professional contexts requires not only providing recommendation results but also offering explanations to persuade users to adopt the suggestions. Taking the task in the primary bond market as an example, where sales staff seek potential investors for bonds, this paper presents the development and deployment of a recommendation system designed for a professional setting. The system provides a set of key features as explanations for its recommendations. In this process, we observe that current explanation methods may select redundant and spurious features, which can undermine the persuasive impact of the explanations. To address this issue, we propose a method named ConciseExplain, which leverages a mask training strategy and gradient descent to directly identify a concise set of features. We conduct experiments on real-world and synthetic datasets. Our method achieves relative improvements of 6.1% and 12.4% over the best-performing baseline on redundant and spurious metrics, respectively. Our method also outperforms the baseline method in online manual evaluations. Moreover, during the one-year official deployment of our system at China Securities Co., Ltd. (a leading brokerage firm in China), we observed a continuous improvement in the accuracy of the recommendation system. This suggests that, with concise explanations, a positive feedback loop might be established between recommendation outcomes and investment decisions.
Retrievers, which form one of the most important recommendation stages, are responsible for efficiently selecting possible positive samples to the later stages under strict latency limitations. Because of this, large-scale systems always rely on approximate calculations and indexes to roughly shrink candidate scale, with a simple ranking model. Most of the existing methods mainly focus on incorporating complicated ranking models. However, index structure is not improved, which also bottlenecks the whole effectiveness. In this paper, we propose a novel index structure: streaming Vector Quantization model, as a new generation of retrieval paradigm. Streaming VQ attaches items with indexes in real time, granting it immediacy. Moreover, through meticulous verification of possible variants, it achieves additional benefits like index balancing and reparability, enabling it to support complicated ranking models as existing approaches. Streaming VQ has been deployed and replaced all major retrievers in Douyin and Douyin Lite, resulting in remarkable user engagement gain.
Representation learning, a task of learning latent vectors to represent entities, is a key task in improving search and recommender systems in web applications. Various representation learning methods have been developed, including graph-based approaches for relationships among entities, sequence-based methods for capturing the temporal evolution of user activities, and content-based models for leveraging text and visual content. However, the development of a unifying framework that integrates these diverse techniques to support multiple applications remains a significant challenge. This paper presents OmniSage, a large-scale representation framework that learns universal representations for a variety of applications at Pinterest. OmniSage integrates graph neural networks with content-based models and user sequence models by employing multiple contrastive learning tasks to effectively process graph data, user sequence data, and content signals. To support the training and inference of OmniSage, we developed an efficient infrastructure capable of supporting Pinterest graphs with billions of nodes. The universal representations generated by OmniSage have significantly enhanced user experiences on Pinterest, leading to an approximate 2.5% increase in sitewide repins (saves) across five applications. This paper highlights the impact of unifying representation learning methods, and we make the model code publicly available https://github.com/pinterest/atg-research/tree/main/omnisage.
Gene selection in high-dimensional genomic data is essential for understanding disease mechanisms and improving therapeutic outcomes. Traditional feature selection methods effectively identify predictive genes but often ignore complex biological pathways and regulatory networks, leading to unstable and biologically irrelevant signatures. Prior approaches, such as Lasso-based methods and statistical filtering, either focus solely on individual gene-outcome associations or fail to capture pathway-level interactions, presenting a key challenge: how to integrate biological pathway knowledge while maintaining statistical rigor in gene selection? To address this gap, we propose a novel two-stage framework that integrates statistical selection with biological pathway knowledge using multi-agent reinforcement learning (MARL). First, we introduce a pathway-guided pre-filtering strategy that leverages multiple statistical methods alongside KEGG pathway information for initial dimensionality reduction. Next, for refined selection, we model genes as collaborative agents in a MARL framework, where each agent optimizes both predictive power and biological relevance. Our framework incorporates pathway knowledge through Graph Neural Network-based state representations, a reward mechanism combining prediction performance with gene centrality and pathway coverage, and collaborative learning strategies using shared memory and a centralized critic component. Extensive experiments on multiple gene expression datasets demonstrate that our approach significantly improves both prediction accuracy and biological interpretability compared to traditional methods.
Toward End-to-End Bearing Fault Diagnosis for Industrial Scenarios with Spiking Neural Networks
PDF ↗This paper explores the application of spiking neural networks (SNNs), known for their low-power binary spikes, to bearing fault diagnosis, bridging the gap between high-performance AI algorithms and real-world industrial scenarios. In particular, we identify two key limitations of existing SNN fault diagnosis methods: inadequate encoding capacity that necessitates cumbersome data preprocessing, and non-spike-oriented architectures that constrain the performance of SNNs. To alleviate these problems, we propose a Multi-scale Residual Attention SNN (MRA-SNN) to simultaneously improve the efficiency, performance, and robustness of SNN methods. By incorporating a lightweight attention mechanism, we have designed a multi-scale attention encoding module to extract multiscale fault features from vibration signals and encode them as spatio-temporal spikes, eliminating the need for complicated preprocessing. Then, the spike residual attention block extracts high-dimensional fault features and enhances the expressiveness of sparse spikes with the attention mechanism for end-to-end diagnosis. In addition, the performance and robustness of MRA-SNN is further enhanced by introducing the lightweight attention mechanism within the spiking neurons to simulate the biological dendritic filtering effect. Extensive experiments on MFPT, JNU, Bearing, and Gearbox benchmark datasets demonstrate that MRA-SNN significantly outperforms existing methods in terms of accuracy, energy consumption, and noise robustness, and is more feasible for deployment in real-world industrial scenarios. Our codes are available at https://github.com/yqding326/MRA-SNN.
Fine-grained Urban Heat Island Effect Forecasting: A Context-aware Thermodynamic Modeling Framework
PDF ↗Climate change and rapid urbanization have led to the Urban Heat Island (UHI) effect, resulting in higher temperatures in metropolitan areas and negatively impacting urban communities. Accurate UHI forecasting is crucial for identifying high-risk periods and locations, especially in cities with vulnerable populations. Current methods are limited by data granularity and inadequate modeling of regional thermodynamics, which affects both accuracy and spatio-temporal granularity. In this paper, we propose DeepUHI, a data-driven context-aware framework for modeling local thermodynamics based on the heat equation, alongside the SeoulTemp dataset, the first multi-modal dataset for UHI effect predictions at the street level. Our framework utilizes a heat decomposition method to represent urban thermodynamics through thermodynamic cycles and thermal flows, effectively integrating urban environmental data. Extensive experiments show that our framework improves accuracy in UHI effect prediction and warning tasks, outperforming leading models. We have integrated DeepUHI into our SeoUHI platform to provide hourly street-level UHI forecasting for Seoul. The code, platform, and dataset are accessible at https://github.com/CityMind-Lab/DeepUHI.
The widespread adoption of mobile devices and data collection technologies has led to an exponential increase in trajectory data, presenting significant challenges in spatio-temporal data mining, particularly for efficient and accurate trajectory retrieval. However, existing methods for trajectory retrieval face notable limitations, including inefficiencies in large-scale data, lack of support for condition-based queries, and reliance on trajectory similarity measures. To address the above challenges, we propose OmniTraj, a generalized and flexible omni-semantic trajectory retrieval framework that integrates four complementary modalities or semantics--raw trajectories, topology, road segments, and regions--into a unified system. Unlike traditional approaches that are limited to computing and processing trajectories as a single modality, OmniTraj designs dedicated encoders for each modality, which are embedded and fused into a shared representation space. This design enables OmniTraj to support accurate and flexible queries based on any individual modality or combination thereof, overcoming the rigidity of traditional similarity-based methods. Extensive experiments on two real-world datasets demonstrate the effectiveness of OmniTraj in handling large-scale data, providing flexible, multi-modality queries, and supporting downstream tasks and applications.
Managing evolving graph data presents substantial challenges in storage and privacy, and training graph neural networks (GNNs) on such data often leads to catastrophic forgetting, impairing performance on earlier tasks. Despite existing continual graph learning (CGL) methods mitigating this to some extent, they rely on centralized architectures and ignore the potential of distributed graph databases to leverage collective intelligence. To this end, we propose Federated Continual Graph Learning (FCGL) to adapt GNNs across multiple evolving graphs under storage and privacy constraints. Our empirical study highlights two core challenges: local graph forgetting (LGF), where clients lose prior knowledge when adapting to new tasks, and global expertise conflict (GEC), where the global GNN exhibits sub-optimal performance in both adapting to new tasks and retaining old ones, arising from inconsistent client expertise during server-side parameter aggregation. To address these, we introduce POWER, a framework that preserves experience nodes with maximum local-global coverage locally to mitigate LGF, and leverages pseudo-prototype reconstruction with trajectory-aware knowledge transfer to resolve GEC. Experiments on various graph datasets demonstrate POWER's superiority over federated adaptations of CGL baselines and vision-centric federated continual learning approaches.
In federated learning (FL), the assumption that datasets from different devices are independent and identically distributed (i.i.d.) often does not hold due to user differences, and the presence of various data modalities across clients makes using a single model impractical. Personalizing certain parts of the model can effectively address these issues by allowing those parts to differ across clients, while the remaining parts serve as a shared model. However, we found that partial model personalization may exacerbate client drift (each client's local model diverges from the shared model), thereby reducing the effectiveness and efficiency of FL algorithms. We propose an FL framework based on the alternating direction method of multipliers (ADMM), referred to as FedAPM, to mitigate client drift. We construct the augmented Lagrangian function by incorporating first-order and second-order proximal terms into the objective, with the second-order term providing fixed correction and the first-order term offering compensatory correction between the local and shared models. Our analysis demonstrates that FedAPM, by using explicit estimates of the Lagrange multiplier, is more stable and efficient in terms of convergence compared to other FL frameworks. We establish the global convergence of FedAPM training from arbitrary initial points to a stationary point, achieving three types of rates: constant, linear, and sublinear, under mild assumptions. We conduct experiments using four heterogeneous and multimodal datasets with different metrics to validate the performance of FedAPM. Specifically, FedAPM achieves faster and more accurate convergence, outperforming the SOTA methods with average improvements of 12.3% in test accuracy, 16.4% in F1 score, and 18.0% in AUC while requiring fewer communication rounds.
With the growing availability of large data repositories within and across organizations, it is becoming feasible to selectively acquire data in the wild for data augmentation, tailored to specific downstream tasks. However, current methodologies concentrate primarily on single-dimensional augmentation tasks for tabular data, such as increasing the number of data points or enriching features. These approaches are not designed for trajectory data, which cannot fully utilize its spatio-temporal characteristics, resulting in suboptimal performance. To address this gap, we present Toast, an iterative framework of task-oriented multi-dimensional augmentation for spatio-temporal trajectory data. Toast supports all three augmentation operations and consists of three core components: (1) augmentation candidate search, which identifies unionable or joinable augmentation candidates in the wild; (2) augmentation candidate selection, which defines and selects the ''best'' augmentations that offer the greatest benefit to downstream tasks; (3) augmentation type selection, which determines the most cost-effective type of augmentation to perform based on a cost model. Extensive experiments on four classical trajectory data mining applications and datasets demonstrate that Toast can enhance model performance by 5%-15% and achieve more than a 2× improvement in efficiency.
Federated learning (FL) enables collaborative training on decentralized data while preserving privacy by avoiding direct data sharing. However, long-tailed data distributions are common in real-world applications, often resulting in biased models with degraded performance. In FL, this issue is further complicated by privacy-preserving constraints and non-IID data, highlighting the importance of federated long-tailed learning (Fed-LT). To tackle the challenges of Fed-LT, we propose Synthetic Feature-based Decoupled training (SFD) method. To improve local training, we introduce Adaptive Bi-Branch Learning (ABBL) to jointly enhance feature representation and decision boundary learning for non-IID long-tailed data. To mitigate global model bias while preserving privacy, we propose Statistically Aligned Feature Synthesis (SAFS) for global classifier fine-tuning. SAFS constructs privacy-preserving synthetic features that approximate the global feature distribution. These synthetic features enable the global classifier to be fine-tuned without requiring clients to share local training data, thereby alleviating the model bias caused by non-IID long-tailed data. Extensive experiments show that SFD effectively addresses the challenges of Fed-LT and achieves superior performance on Fed-LT datasets.
Dynamic link prediction, which aims to predict future interactions based on historical temporal graph evolution, plays a pivotal role in applications such as social networks, recommendation systems, and patent trend prediction. However, existing research predominantly validates model performance under a single data distribution characterized by high-density repeated interactions, which leads to an insufficient exploration of higher-order node relationships. In contrast, we argue that dynamic graphs exhibit diverse application potential, where the occurrence of future interactions is not exclusively determined by historical repeated events, but rather by higher-order positional dependencies. To fill this gap, we classify the data into two scenarios with dense and sparse repeated interactions, and encode the temporal positions of neighbors relative to the event center to better guide hierarchical neighborhood aggregation. Building upon this, we propose PANet, a position-aware dynamic graph network that captures the temporal positioning of nodes in evolving interactions. Firstly, we define the spatial and temporal position of neighbors based on their distance to the event center and their occurrence frequency. Secondly, we employ a hierarchical paired-position encoding approach to compute the temporal position representation for each neighbor. Last, we leverage the relative positions and timestamps of neighbors to guide the aggregation of neighborhood information, and derive the final node representation through an attention-based network. Experiments conducted on eight datasets across two distributions validate the effectiveness and efficiency of the proposed method. Moreover, the significant performance improvement achieved by incorporating the position-aware neighbor aggregation into existing dynamic graph methods further demonstrates its versatility.
Cuff-KT: Tackling Learners' Real-time Learning Pattern Adjustment via Tuning-Free Knowledge State Guided Model Updating
PDF ↗Knowledge Tracing (KT) is a core component of Intelligent Tutoring Systems, modeling learners' knowledge state to predict future performance and provide personalized learning support. Traditional KT models assume that learners' learning abilities remain relatively stable over short periods or change in predictable ways based on prior performance. However, in reality, learners' abilities change irregularly due to factors like cognitive fatigue, motivation, and external stress--a task introduced, which we refer to as Real-time Learning Pattern Adjustment (RLPA). Existing KT models, when faced with RLPA, lack sufficient adaptability, because they fail to timely account for the dynamic nature of different learners' evolving learning patterns. Current strategies for enhancing adaptability rely on retraining, which leads to significant overfitting and high time overhead issues. To address this, we propose Cuff-KT, comprising a controller and a generator. The controller assigns value scores to learners, while the generator generates personalized parameters for selected learners. Cuff-KT controllably adapts to data changes fast and flexibly without fine-tuning. Experiments on five datasets from different subjects demonstrate that Cuff-KT significantly improves the performance of five KT models with different structures under intra- and inter-learner shifts, with an average relative increase in AUC of 10% and 4%, respectively, at a negligible time cost, effectively tackling RLPA task. Our code and datasets are fully available at https://github.com/zyy-2001/Cuff-KT.
Trajectory representation learning (TRL) maps trajectories to vector embeddings and facilitates tasks such as trajectory classification and similarity search. State-of-the-art (SOTA) TRL methods transform raw GPS trajectories to grid or road trajectories to capture high-level travel semantics, i.e., regions and roads. However, they lose fine-grained spatial-temporal details as multiple GPS points are grouped into a single grid cell or road segment. To tackle this problem, we propose the BLU rred Encoding method, dubbed BLUE, which gradually reduces the precision of GPS coordinates to create hierarchical patches with multiple levels. The low-level patches are small and preserve fine-grained spatial-temporal details, while the high-level patches are large and capture overall travel patterns. To complement different patch levels with each other, our BLUE is an encoder-decoder model with a pyramid structure. At each patch level, a Transformer is used to learn the trajectory embedding at the current level, while pooling prepares inputs for the higher level in the encoder, and up-resolution provides guidance for the lower level in the decoder. BLUE is trained using the trajectory reconstruction task with the MSE loss. We compare BLUE with 8 SOTA TRL methods for 3 downstream tasks, the results show that BLUE consistently achieves higher accuracy than all baselines, outperforming the best-performing baselines by an average of 30.90%. Our code is available at https://github.com/slzhou-xy/BLUE.
CrossLinear: Plug-and-Play Cross-Correlation Embedding for Time Series Forecasting with Exogenous Variables
PDF ↗Time series forecasting with exogenous variables is a critical emerging paradigm that presents unique challenges in modeling dependencies between variables. Traditional models often struggle to differentiate between endogenous and exogenous variables, leading to inefficiencies and overfitting. In this paper, we introduce CrossLinear, a novel Linear-based forecasting model that addresses these challenges by incorporating a plug-and-play cross-correlation embedding module. This lightweight module captures the dependencies between variables with minimal computational cost and seamlessly integrates into existing neural networks. Specifically, it captures time-invariant and direct variable dependencies while disregarding time-varying or indirect dependencies, thereby mitigating the risk of overfitting in dependency modeling and contributing to consistent performance improvements. Furthermore, CrossLinear employs patch-wise processing and a global linear head to effectively capture both short-term and long-term temporal dependencies, further improving its forecasting precision. Extensive experiments on 12 real-world datasets demonstrate that CrossLinear achieves superior performance in both short-term and long-term forecasting tasks. The ablation study underscores the effectiveness of the cross-correlation embedding module. Additionally, the generalizability of this module makes it a valuable plug-in for various forecasting tasks across different domains. Codes are available at https://github.com/mumiao2000/CrossLinear.
Cross-domain sequential recommendation (CDSR) aims to enhance recommendation performance in a target domain by leveraging user sequential preferences from a source domain. As most advanced methods employ various neural networks to model item-ID sequences, they often struggle with a sparse data. Recent text-based CDSR approaches utilize rich text information to facilitate knowledge transfer across domains but still encounter three main challenges: (i) how to capture domain-shared semantic relationships; (ii) how to effectively integrate semantic intents and behavior intents; and (iii) how to design sufficient supervision signals for model training. In this paper, we propose a novel text-enhanced CDSR solution, i.e., contrastive text-enhanced Transformer (CTT), which jointly learns user preferences from semantic and behavioral information. Specifically, our CTT is a dual-view structure, including an inter-domain preference view and an intra-domain preference view. In the former, we design a domain-shared cross-attention mechanism to encode semantic correlations across different domains by an attention map, which is then utilized for cross-domain semantic intent learning. In the latter, we extract fine-grained behavior intents from item-ID sequences and coarse-grained semantic intents from item-text sequences, thereby capturing high-quality intra-domain preferences. Our CTT integrates these two types of preferences to predict the next item for each user. Moreover, we propose a semantic-behavior contrastive learning approach to enhance cross-domain preference learning and knowledge transfer. Extensive experiments on three real-world datasets demonstrate that our CTT outperforms the state-of-the-art models by an average of 5.86% on HR@5. The scripts for data preprocessing and conducting experiments, the parameter configurations, and the source codes of our CTT and all baselines are available at https://github.com/XinbeiCai/CTT.
A Fast and Accurate Block Compression Solution for Spatiotemporal Kernel Density Visualization
PDF ↗Spatiotemporal Kernel Density Visualization (STKDV) has been widely used across various domains in geospatial analysis, e.g., urban planning, traffic/traffic accident hotspot analysis, crime hotspot analysis, and disease spread modeling. However, STKDV is a computationally expensive tool, which has been complained by many domain experts. Although many recent solutions, including the sliding-window-based solution (SWS) and the prefix-matrix-based solution (PREFIX), have been proposed for improving the efficiency of generating an exact STKDV, these solutions still cannot be scalable to handle large-scale location datasets. To tackle this efficiency issue, we propose the pioneering block compression solution, called COMP, which can compress (or represent) a location dataset by a small amount of blocks. By combining COMP with the existing exact solutions, i.e., SWS and PREFIX, we show that COMPSWS and COMPPREFIX can generate approximate STKDV with an ε-absolute error guarantee based on properly tuning the block size. Experimental results on four large-scale location datasets (up to 6.782 million data points) also verify that COMPSWS and COMP_PREFIX can achieve speedups of 4.1x to 677.16x and 1.45x to 143.52x compared with SWS and PREFIX, respectively, without degrading the visualization results. The code of this paper can be found in https://github.com/YovelaZ/COMP.
MedDiTPro: A Prompt-Guided Diffusion Transformer for Multimodal Longitudinal Medical Data Synthesis
PDF ↗Diffusion models have recently emerged as a state-of-the-art approach for synthetic Electronic Health Record (EHR) generation, offering superior fidelity and diversity over traditional generative models. However, existing diffusion-based methods struggle with unique challenges: limited representation learning and modality utilization, where they fail to explicitly capture inter-modality dependencies and fine-grained code-level interactions, and constrained adaptability due to reliance on U-Net-based architectures, which are not well-suited for handling the heterogeneous and evolving nature of EHR data. Furthermore, current evaluation paradigms rely on either perplexity-based sequence modeling or global distributional measures, lacking robustness in assessing both intra-visit code relationships and inter-visit temporal patterns. To address these limitations, we propose MedDiTPro, a diffusion transformer-based framework that enhances multimodal EHR generation by integrating structured modality-aware guidance. Through a unified transformer for intra-visit representation learning, a modality-specific and datawise prompt learner, and a diffusion transformer with structured guidance, MedDiTPro achieves state-of-the-art performance in generating diverse and clinically meaningful synthetic records. Extensive experiments on publicly available datasets demonstrate that MedDiTPro achieves state-of-the-art fidelity, privacy preservation, and utility.