Rapid urbanization has significantly escalated traffic congestion, underscoring the need for advanced congestion prediction services to bolster intelligent transportation systems. As one of the world's largest ride-hailing platforms, DiDi places great emphasis on the accuracy of congestion prediction to enhance the effectiveness and reliability of their real-time services, such as travel time estimation and route planning. Despite numerous efforts have been made on congestion prediction, most of them fall short in handling heterogeneous and dynamic spatio-temporal dependencies (e.g., periodic and non-periodic congestions), particularly in the presence of noisy and incomplete traffic data. In this paper, we introduce a Congestion Prediction Mixture-of-Experts, CP-MoE, to address the above challenges. We first propose a sparsely-gated Mixture of Adaptive Graph Learners (MAGLs) with congestion-aware inductive biases to improve the model capacity for efficiently capturing complex spatio-temporal dependencies in varying traffic scenarios. Then, we devise two specialized experts to help identify stable trends and periodic patterns within the traffic data, respectively. By cascading these experts with MAGLs, CP-MoE delivers congestion predictions in a more robust and interpretable manner. Furthermore, an ordinal regression strategy is adopted to facilitate effective collaboration among diverse experts. Extensive experiments on real-world datasets demonstrate the superiority of our proposed method compared with state-of-the-art spatio-temporal prediction models. More importantly, CP-MoE has been deployed in DiDi to improve the accuracy and reliability of the travel time estimation system.
论文检索
输入标题、作者或关键词，从 642 篇学术成果中精准定位
Deep Recommender Systems (DRS) are increasingly dependent on a large number of feature fields for more precise recommendations. Effective feature selection methods are consequently becoming critical for further enhancing the accuracy and optimizing storage efficiencies to align with the deployment demands. This research area, particularly in the context of DRS, is nascent and faces three core challenges. Firstly, variant experimental setups across research papers often yield unfair comparisons, obscuring practical insights. Secondly, the existing literature's lack of detailed analysis on selection attributes, based on large-scale datasets and a thorough comparison among selection techniques and DRS backbones, restricts the generalizability of findings and impedes deployment on DRS. Lastly, research often focuses on comparing the peak performance achievable by feature selection methods. This approach is typically computationally infeasible for identifying the optimal hyperparameters and overlooks evaluating the robustness and stability of these methods. To bridge these gaps, this paper presents ERASE, a comprehensive bEnchmaRk for feAture SElection for DRS. ERASE comprises a thorough evaluation of eleven feature selection methods, covering both traditional and deep learning approaches, across four public datasets, private industrial datasets, and a real-world commercial platform, achieving significant enhancement. Our code is available online for ease of reproduction.
Online controlled experiments are a crucial tool to allow for confident decision-making in technology companies. A North Star metric is defined (such as long-term revenue or user retention), and system variants that statistically significantly improve on this metric in an A/B-test can be considered superior. North Star metrics are typically delayed and insensitive. As a result, the cost of experimentation is high: experiments need to run for a long time, and even then, type-II errors (i.e. false negatives) are prevalent. We propose to tackle this by learning metrics from short-term signals that directly maximise the statistical power they harness with respect to the North Star. We show that existing approaches are prone to overfitting, in that higher average metric sensitivity does not imply improved type-II errors, and propose to instead minimise the p-values a metric would have produced on a log of past experiments. We collect such datasets from two social media applications with over 160 million Monthly Active Users each, totalling over 153 A/B-pairs. Empirical results show that we are able to increase statistical power by up to 78% when using our learnt metrics stand-alone, and by up to 210% when used in tandem with the North Star. Alternatively, we can obtain constant statistical power at a sample size that is down to 12% of what the North Star requires, significantly reducing the cost of experimentation.
As the focus of space exploration shifts from national agencies to private companies, the interest in space industry has been steadily increasing. With the increasing number of satellites, the risk of collisions between satellites and space debris has escalated, potentially leading to significant property and human losses. Therefore, accurately modeling the orbit is critical for satellite operations. In this work, we propose the Decomposed Attention Segment Recurrent Neural Network (DASR) model, adding two key components, Multi-Head Attention and Tensor Train Decomposition, to SegRNN for orbit prediction. The DASR model applies Multi-Head Attention before segmenting at input data and before the input of the GRU layers. In addition, Tensor Train (TT) Decomposition is applied to the weight matrices of the Multi-Head Attention in both the encoder and decoder. For evaluation, we use three real-world satellite datasets from the Korea Aerospace Research Institute (KARI), which are currently operating: KOMPSAT-3, KOMPSAT-3A, and KOMPSAT-5 satellites. Our proposed model demonstrates superior performance compared to other SOTA baseline models. We demonstrate that our approach has 94.13% higher predictive performance than the second-best model in the KOMPSAT-3 dataset, 89.79% higher in the KOMPSAT-3A dataset, and 76.71% higher in the KOMPSAT-5 dataset.
Product assortment selection is a critical challenge facing physical retailers. Effectively aligning inventory with the preferences of shoppers can increase sales and decrease out-of-stocks. However, in real-world settings the problem is challenging due to the combinatorial explosion of product assortment possibilities. Consumer preferences are typically heterogeneous across space and time, making inventory-preference alignment challenging. Additionally, existing strategies rely on syndicated data, which tends to be aggregated, low resolution, and suffer from high latency. To solve these challenges, we introduce a real-time recommendation system, which we call EdgeRec3D. Our system utilizes recent advances in 3D computer vision for perception and automatic, fine grained sales estimation. These perceptual components run on the edge of the network and facilitate real-time reward signals. Additionally, we develop a Bayesian payoff model to account for noisy estimates from 3D LIDAR data. We rely on spatial clustering to allow the system to adapt to heterogeneous consumer preferences, and a graph-based candidate generation algorithm to address the combinatorial search problem. We test our system in real-world stores across two, 6-8 week A/B tests with beverage products and demonstrate a 35% and 27% increase in sales respectively. Finally, we monitor the deployed system for a period of 28 weeks with an observational study and show a 9.4% increase in sales.
The unsecured personal loan (UPL) market is a multi-billion dollar market where numerous financial institutions compete. Due to the development of online banking, loan applicants start to compare numerous loan products. They aim for high loan limits and low interest rates. Since loan applicants have a desired loan amount, institutions instead focus on adjusting interest rates. Despite the importance of determining optimal interest strategies, institutions have traditionally relied on heuristic methods by human experts to set interest rates. This is done by adding a target return on assets (ROA) to the applicant's expected default probability predicted by a credit scoring system (CSS) such as the FICO score. We conceptualize the UPL market dynamics as a repeated auction scenario, where loan applicants (akin to sellers) seek the lowest interest rates, while financial institutions (akin to bidders) aim to maximize profits through higher interest rates. To the best of our knowledge, this is the first time anyone has approached the UPL market through the viewpoint of a repeated auction. While there are several research done in learning to bid in repeated auctions, those works cannot be directly applied to the UPL market due to the lack of any feedback about other bidders' strategies and the need to satisfy the bidder's target loan volume and profit variance. We present an algorithm named AutoInterest, which is a modification of the dual gradient descent algorithm. In addition, we provide a framework to evaluate interest rate bidding strategies on a benchmark dataset and the credit bureau dataset of actual loan applicants in South Korea. We evaluate AutoInterest on this framework and show higher cumulative profit compared to other common online algorithms and the current fixed strategy used by real institutions.
Personalised Drug Identifier for Cancer Treatment with Transformers using Auxiliary Information
PDF ↗Cancer remains a global challenge due to its growing clinical and economic burden. Its uniquely personal manifestation, which makes treatment difficult, has fuelled the quest for personalized treatment strategies. Thus, genomic profiling is increasingly becoming part of clinical diagnostic panels. Effective use of such panels requires accurate drug response prediction (DRP) models, which are challenging to build due to limited labelled patient data. Previous methods to address this problem have used various forms of transfer learning. However, they do not explicitly model the variable length sequential structure of the list of mutations in such diagnostic panels. Further, they do not utilize auxiliary information (like patient survival) for model training. We address these limitations through a novel transformer-based method, which surpasses the performance of state-of-the-art DRP models on benchmark data. Code for our method is available at https://github.com/CDAL-SOC/PREDICT-AI. We also present the design of a treatment recommendation system (TRS), which is currently deployed at the National University Hospital, Singapore and is being evaluated in a clinical trial. We discuss why the recommended drugs and their predicted scores alone, obtained from DRP models, are insufficient for treatment planning. Treatment planning for complex cancer cases, in the face of limited clinical validation, requires assessment of many other factors, including several indirect sources of evidence on drug efficacy. We discuss key lessons learnt on model validation and use of indirect supporting evidence to build clinicians' trust and aid their decision making.
Explainable and Interpretable Forecasts on Non-Smooth Multivariate Time Series for Responsible Gameplay
PDF ↗Multi-variate Time Series (MTS) forecasting has made large strides (with very negligible errors) through recent advancements in neural networks, e.g., Transformers. However, in critical situations like predicting gaming overindulgence that affects one's mental well-being; an accurate forecast without a contributing evidence (explanation) is irrelevant. Hence, it becomes important that the forecasts are Interpretable - intermediate representation of the forecasted trajectory is comprehensible; as well as Explainable - attentive input features and events are accessible for a personalized and timely intervention of players at risk. While the contributing state of the art research on interpretability primarily focuses on temporally-smooth single-process driven time series data, our online multi-player gameplay data demonstrates intractable temporal randomness due to intrinsic orthogonality between player's game outcome and their intent to engage further. We introduce a novel deep Actionable Forecasting Network (AFN), which addresses the inter-dependent challenges associated with three exclusive objectives - 1) forecasting accuracy; 2) smooth comprehensible trajectory and 3) explanations via multi-dimensional input features while tackling the challenges introduced by our non-smooth temporal data, together in one single solution. AFN establishes a new benchmark via: (i) achieving 25% improvement on the MSE of the forecasts on player data in comparison to the SOM-VAE based SOTA networks; (ii) attributing unfavourable progression of a player's time series to a specific future time step(s), with the premise of eliminating near-future overindulgent player volume by over 18% with player specific actionable inputs feature(s) and (iii) proactively detecting over 23% (100% jump from SOTA) of the to-be overindulgent, players on an average, 4 weeks in advance.
Lifelong sequential modeling (LSM) has significantly advanced recommendation systems on social media platforms. Diverging from single-domain LSM, cross-domain LSM involves modeling lifelong behavior sequences from a source domain to a different target domain. In this paper, we propose the Lifelong Cross Network (LCN), a novel approach for cross-domain LSM. LCN features a Cross Representation Production (CRP) module that utilizes contrastive loss to improve the learning of item embeddings, effectively bridging items across domains. This is important for enhancing the retrieval of relevant items in cross-domain lifelong sequences. Furthermore, we propose the Lifelong Attention Pyramid (LAP) module, which contains three cascading attention levels. By adding an intermediate level and integrating the results from all three levels, the LAP module can capture a broad spectrum of user interests and ensure gradient propagation throughout the sequence. The proposed LAP can also achieve remarkable consistency across attention levels, making it possible to further narrow the candidate item pool of the top level. This allows for the use of advanced attention techniques to effectively mitigate the impact of the noise in cross-domain sequences and improve the non-linearity of the representation, all while maintaining computational efficiency. Extensive experiments conducted on both a public dataset and an industrial dataset from the WeChat Channels platform reveal that the LCN outperforms current methods in terms of prediction accuracy and online performance metrics.
Independent and identically distributed (i.i.d.) data is essential to many data analysis and modeling techniques. In the medical domain, collecting data from multiple sites or institutions is a common strategy that guarantees sufficient clinical diversity, determined by the decentralized nature of medical data. However, data from various sites are easily biased by the local environment or facilities, thereby violating the i.i.d. rule. A common strategy is to harmonize the site bias while retaining important biological information. The ComBat is among the most popular harmonization approaches and has recently been extended to handle distributed sites. However, when faced with situations involving newly joined sites in training or evaluating data from unknown/unseen sites, ComBat lacks compatibility and requires retraining with data from all the sites. The retraining leads to significant computational and logistic overhead that is usually prohibitive. In this work, we develop a novel Cluster ComBat harmonization algorithm, which leverages cluster patterns of the data in different sites and greatly advances the usability of ComBat harmonization. We use extensive simulation and real medical imaging data from ADNI to demonstrate the superiority of the proposed approach. Our codes are provided in https://github.com/illidanlab/distributed-cluster-harmonization.
Uplift modeling has been widely employed in online marketing by predicting the response difference between the treatment and control groups, so as to identify the sensitive individuals toward interventions like coupons or discounts. Compared with traditional conversion uplift modeling,revenue uplift modeling exhibits higher potential due to its direct connection with the corporate income. However, previous works can hardly handle the continuous long-tail response distribution in revenue uplift modeling. Moreover, they have neglected to optimize the uplift ranking among different individuals, which is actually the core of uplift modeling. To address such issues, in this paper, we first utilize the zero-inflated lognormal (ZILN) loss to regress the responses and customize the corresponding modeling network, which can be adapted to different existing uplift models. Then, we study the ranking-related uplift modeling error from the theoretical perspective and propose two tighter error bounds as the additional loss terms to the conventional response regression loss. Finally, we directly model the uplift ranking error for the entire population with a listwise uplift ranking loss. The experiment results on offline public and industrial datasets validate the effectiveness of our method for revenue uplift modeling. Furthermore, we conduct large-scale experiments on a prominent online fintech marketing platform, Tencent FiT, which further demonstrates the superiority of our method in real-world applications.
Link prediction, determining if a relation exists between two entities, is an essential task in the analysis of heterogeneous graphs with diverse entities and relations. Despite extensive research in link prediction, most existing works focus on predicting the relation type between given pairs of entities. However, it is almost impractical to check every entity pair when trying to find most hidden relations in a billion-scale heterogeneous graph due to the billion squared number of possible pairs. Meanwhile, most methods aggregate information at the node level, potentially leading to the loss of direct connection information between the two nodes. In this paper, we introduce Paths2Pair, a novel framework to address these limitations for link prediction in billion-scale commercial heterogeneous graphs. (i) First, it selects a subset of reliable entity pairs for prediction based on relevant meta-paths. (ii) Then, it utilizes various types of content information from the meta-paths between each selected entity pair to predict whether a target relation exists. We first evaluate our Paths2Pair based on a large-scale dataset, and results show Paths2Pair outperforms state-of-the-art baselines significantly. We then deploy our Paths2Pair on JD Logistics, one of the largest logistics companies in the world, for business expansion. The uncovered relations by Paths2Pair have helped JD Logistics identify 108,709 contacts to attract new company customers, resulting in an 84% increase in the success rate compared to the state-of-the-practice solution, demonstrating the practical value of our framework. We have released the code of our framework at https://github.com/JQHang/Paths2Pair.
This paper introduces FedSecurity, an end-to-end benchmark that serves as a supplementary component of the FedML library for simulating adversarial attacks and corresponding defense mechanisms in Federated Learning (FL). FedSecurity eliminates the need for implementing the fundamental FL procedures, e.g., FL training and data loading, from scratch, thus enables users to focus on developing their own attack and defense strategies. It contains two key components, including FedAttacker that conducts a variety of attacks during FL training, and FedDefender that implements defensive mechanisms to counteract these attacks. FedSecurity has the following features: i) It offers extensive customization options to accommodate a broad range of machine learning models (e.g., Logistic Regression, ResNet, and GAN) and FL optimizers (e.g., FedAVG, FedOPT, and FedNOVA); ii) it enables exploring the effectiveness of attacks and defenses across different datasets and models; and iii) it supports flexible configuration and customization through a configuration file and some APIs. We further demonstrate FedSecurity's utility and adaptability through federated training of Large Language Models (LLMs) to showcase its potential on a wide range of complex applications.
As a two-sided marketplace, Airbnb brings together hosts who own listings for rent with prospective guests from around the globe. Results from a guest's search for listings are displayed primarily through two interfaces: (1) as a list of rectangular cards that contain on them the listing image, price, rating, and other details, referred to as list-results (2) as oval pins on a map showing the listing price, called map-results. Both these interfaces, since their inception, have used the same ranking algorithm that orders listings by their booking probabilities and selects the top listings for display. But some of the basic assumptions underlying ranking, built for a world where search results are presented as lists, simply break down for maps. This paper describes how we rebuilt ranking for maps by revising the mathematical foundations of how users interact with search results. Our iterative and experiment-driven approach led us through a path full of twists and turns, ending in a unified theory for the two interfaces. Our journey shows how assumptions taken for granted when designing machine learning algorithms may not apply equally across all user interfaces, and how they can be adapted. The net impact was one of the largest improvements in user experience for Airbnb which we discuss as a series of experimental validations.
Music composition and analysis is an inherently creative task, involving a combination of heart and mind. However, the vast majority of algorithmic music models completely ignore the "heart" component of music, resulting in output that often lacks the rich emotional direction found in human-composed music. Models that try to incorporate musical sentiment rely on a "valence-arousal" model, which insufficiently characterizes emotion in two dimensions. Furthermore, existing methods typically adopt a black-box, music agnostic approach, treating music-theoretical and sentimental understanding as a by-product that can be inferred given sufficient data. In this study, we introduce two major novel elements: a nuanced mixture-based representation for musical sentiment, including a web tool to gather data, as well as a sentiment- and theory-driven harmonization model, SentHYMNent. SentHYMNent employs a novel Hidden Markov Model based on both key and chord transitions, as well as sentiment mixtures, to provide a probabilistic framework for learning key modulations and chordal progressions from a given melodic line and sentiment. Furthermore, our approach leverages compositional principles, resulting in a simpler model that significantly reduces computational burden and enhances interpretability compared to current state-of-the-art algorithmic harmonization methods. Importantly, as shown in our experiments, these improvements do not come at the expense of harmonization quality. We also provide a web app where users can upload their own melodies for SentHYMNent to harmonize.
Auto-bidding plays a crucial role in facilitating online advertising by automatically providing bids for advertisers. Reinforcement learning (RL) has gained popularity for auto-bidding. However, most current RL auto-bidding methods are modeled through the Markovian Decision Process (MDP), which assumes the Markovian state transition. This assumption restricts the ability to perform in long horizon scenarios and makes the model unstable when dealing with highly random online advertising environments. To tackle this issue, this paper introduces AI-Generated Bidding (AIGB), a novel paradigm for auto-bidding through generative modeling. In this paradigm, we propose DiffBid, a conditional diffusion modeling approach for bid generation. DiffBid directly models the correlation between the return and the entire trajectory, effectively avoiding error propagation across time steps in long horizons. Additionally, DiffBid offers a versatile approach for generating trajectories that maximize given targets while adhering to specific constraints. Extensive experiments conducted on the real-world dataset and online A/B test on Alibaba advertising platform demonstrate the effectiveness of DiffBid, achieving 2.81% increase in GMV and 3.36% increase in ROI.
Logistics advertising is an emerging task in online-to-offline logistics systems, where logistics companies expand parcel shipping services to new users through advertisements on shopping websites. Compared to existing online e-commerce advertising, logistics advertising has two significant new characteristics: (i) the complex factors in logistics advertising considering both users' offline logistics preference and online purchasing profiles; and (ii) data sparsity and mutual relations among multiple steps due to longer advertising conversion processes. To address these challenges, we design MCAC, a Multi-task Conditional Attention network-based logistics advertising Conversion prediction framework, which consists of (i) an offline shipping preference extraction model to extract the user's offline logistics preference from historical shipping records, and (ii) a multi-task conditional attention-based conversion rate prediction module to model mutual relations among multiple steps in logistics advertising conversion processes. We evaluate and deploy MCAC on one of the largest e-commerce platforms in China for logistics advertising. Extensive offline experiments show that our method outperforms state-of-the-art baselines in various metrics. Moreover, the conversion rate prediction results of large-scale online A/B testing show that MCAC achieves a 15.22% improvement compared to existing industrial practices, which demonstrates the effectiveness of the proposed framework.
While intelligent virtual assistants like Siri, Alexa, and Google Assistant have become ubiquitous in modern life, they still face limitations in their ability to follow multi-step instructions and accomplish complex goals articulated in natural language. However, recent breakthroughs in large language models (LLMs) show promise for overcoming existing barriers by enhancing natural language processing and reasoning capabilities. Though promising, applying LLMs to create more advanced virtual assistants still faces challenges like ensuring robust performance and handling variability in real-world user commands. This paper proposes a novel LLM-based virtual assistant that can automatically perform multi-step operations within mobile apps based on high-level user requests. The system represents an advance in assistants by providing an end-to-end solution for parsing instructions, reasoning about goals, and executing actions. LLM-based Process Automation (LLMPA) has modules for decomposing instructions, generating descriptions, detecting interface elements, predicting next actions, and error checking. Experiments demonstrate the system completing complex mobile operation tasks in Alipay based on natural language instructions. This showcases how large language models can enable automated assistants to accomplish real-world tasks. The main contributions are the novel LLMPA architecture optimized for app process automation, the methodology for applying LLMs to mobile apps, and demonstrations of multi-step task completion in a real-world environment. Notably, this work represents the first real-world deployment and extensive evaluation of a large language model-based virtual assistant in a widely used mobile application with an enormous user base numbering in the hundreds of millions.
Customer segmentation plays a crucial role in credit risk assessment by dividing users into specific risk levels based on their credit scores. Previous methods fail to comprehensively consider the stability in the segmentation process, resulting in frequent changes and inconsistencies in users' risk levels over time. This increases potential risks to a company. To this end, this paper at first introduces and formalizes the concept of stability regret in the segmentation process. However, evaluating stability is challenging due to its black-box nature and the computational burden posed by vast user data sets. To address these challenges, this paper proposes a large-scale stable customer segmentation approach named LASCA. LASCA consists of two phases: high-quality dataset construction (HDC) and reliable data-driven optimization (RDO). Specifically, HDC utilizes an evolutionary algorithm to collect high-quality binning solutions. RDO subsequently builds a reliable surrogate model to search for the most stable binning solution based on the collected dataset. Extensive experiments conducted on real-world large-scale datasets (up to 0.8 billion) show that LASCA surpasses the state-of-the-art binning methods in finding the most stable binning solution. Notably, HDC greatly enhances data quality by 50%. RDO efficiently discovers more stable binning solutions with a 36% improvement in stability, accelerating the optimization process by 25 times via data-driven evaluation. Currently, LASCA has been successfully deployed in the large-scale credit risk assessment system of Alipay.
Freight transportation marketplace rates are typically challenging to forecast accurately. In this work, we have developed a novel statistical technique based on signature transforms and have built a predictive and adaptive model to forecast these marketplace rates. Our technique is based on two key elements of the signature transform: one being its universal nonlinearity property, which linearizes the feature space and hence translates the forecasting problem into linear regression, and the other being the signature kernel, which allows for comparing computationally efficiently similarities between time series data. Combined, it allows for efficient feature generation and precise identification of seasonality and regime switching in the forecasting process. An algorithm based on our technique has been deployed by Amazon trucking operations, with far superior forecast accuracy and better interpretability versus commercially available industry models, even during the COVID-19 pandemic and the Ukraine conflict. Furthermore, our technique is able to capture the influence of business cycles and the heterogeneity of the marketplace, improving prediction accuracy by more than fivefold, with an estimated annualized saving of \50 million.