On-demand Food Delivery (OFD) platforms, such as Ele.me and Meituan, have transformed daily life by offering convenient ordering services. However, challenges remain in understanding user intentions and processing product-related text information. Existing NLP models, while advanced in general tasks, are less effective for OFD-specific needs due to data scarcity and high computational costs. This paper introduces FoodInstruct, a Chinese dataset with 1.6 million examples across 12 OFD-related NLP tasks, and FoodGPT, a domain-specific large language model. We propose an efficient reinforcement post-training framework that combines Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Proximal Policy Optimization (PPO) with an additional rule-based reward signal. The resulting foundation model, FoodGPT, enhances model performance while minimizing resource consumption. Experimental results demonstrate that FoodGPT outperforms general models, such as Qwen 2.5, Llama3.1 and DeepSeek-LLM, on OFD tasks, using fewer data and training iterations. The model has already been deployed across numerous applications within the company. For instance, it achieved a 0.57% increase in click-through rate and a 0.32% increase in user visit-to-purchase rate in the ITG online experiment. The core contributions are now publicly accessible at https://huggingface.co/elemenlp.
论文检索
输入标题、作者或关键词，从 844 篇学术成果中精准定位
Improved Accuracy, Declining Orders: Uncovering the Vicious Competition Trap in Multi-Domain Recommendation
PDF ↗E-commerce recommendation platforms often operate across multiple domains, giving rise to Multi-Domain Recommendation (MDR) methods. These methods leverage auxiliary information from other domains to capture comprehensive user preferences and enhance recommendation accuracy. However, cross-domain knowledge transfer often causes domains to capture similar preferences. Additionally, most MDR models optimize for next-item prediction, which promotes greedy strategies focused on immediate gains within individual domains while neglecting the platform-wide profit. These two factors result in vicious competition, as our empirical analysis reveals: Domains recommend overlapping item lists, competing for the same pool of user attention and platform traffic, reducing the overall orders. To address this, we propose the Direct Preference Optimization from Manual Rules (DPOMR) training paradigm for MDR. Inspired by Reinforcement Learning from Human Feedback (RLHF), DPOMR learns user preferences from pairwise preference data generated through manual rules rooted in business expertise. Furthermore, we design a variant of Direct Preference Optimization (DPO) to bypass expensive RL training. Extensive experiments demonstrate the advantages of DPOMR, including: (1) achieving SOTA performance in boosting overall orders(2) showing compatibility with various backbones, and (3) offering robust extendibility to integrate diverse manual rules tailored to specific business needs.
Spotify, a large-scale multimedia platform, attracts over 675 million monthly active users who collectively consume millions of hours of music, podcasts, audiobooks, and video content. This diverse content consumption pattern introduces unique challenges for computational advertising, which must effectively integrate a variety of ad modalities including audio, video, and display-within a single user experience. Traditional ad recommendation models, primarily designed for foregrounded experiences, often struggle to reconcile the platform's inherent audio-centrality with the demands of optimizing ad performance across multiple formats and modalities. To overcome these challenges, we introduce Cross-modal Adaptive Mixture-of-Experts (CAMoE), a novel framework for optimizing click-through rate (CTR) prediction in both audio-centric and multi-modal settings. CAMoE enhances traditional mixture-of-experts (MoE) models by incorporating modality-aware task grouping, adaptive loss masking, and deep-cross networks (DCN) to capture complex feature interactions within a multi-modal ad ecosystem. Through extensive ablation studies, we demonstrate that this approach achieves near Pareto-optimal performance across audio, video, and display ad formats, significantly improving AUC-PR compared to conventional single-task and content-based multi-task learning baselines. When deployed at scale on Spotify's ad serving platform, CAMoE delivered substantial gains, yielding a 14.5% increase in CTR for audio ads, a 1.3% increase for video ads, and a 4.8% reduction in expected cost-per-click (eCPC) for audio slots.
Industry-grade ML models are carefully designed to meet rapidly evolving serving constraints, which requires significant resources for model development. In this paper, we propose MatTA, a framework for training multiple accurate Student models using a novel Teacher-TA-Student recipe. TA models are larger versions of the Student models with higher capacity, and thus allow Student models to better relate to the Teacher model and also bring in more domain-specific expertise. Furthermore, multiple accurate Student models can be extracted from the TA model. Therefore, despite only one training run, our methodology provides multiple servable options to trade off accuracy for lower serving cost. We demonstrate the proposed method, MatTA, on proprietary datasets and models. Its practical efficacy is underscored by live A/B tests within a production ML system, demonstrating 20% improvement on a key metric. We also demonstrate our method on GPT-2 Medium, a public model, and achieve relative improvements of over 24% on SAT Math and over 10% on the LAMBADA benchmark.
Characterizing the Risk of Atrial Fibrillation in Cardiac Patients with Exceptional Electrocardiogram Phenotypes
PDF ↗We provide a transparent method to characterize Atrial Fibrillation (AF) caused by cardiac surgery, using Electrocardiogram (ECG) phenotypes. Current practice in the hospital is reactive rather than preventive and is based on a third party's proprietary alarms on vitals. Assistance with detection and prediction methods often lacks sufficient insights into their decisions toward the users, i.e., the hospital workers. This aspect is necessary to gain the trust of medical workers and patients in the decisions that are made. Our objective of transparently identifying risk factors for AF helps experts increase their understanding of the problem, and assists in decision-making about administering preventive medication to risk groups. With the deployment of the Exceptional Model Mining (EMM) framework on AF-related ECG phenotypes, we introduce a transparent and actionable method that assists the hospital in preventive treatment. We find several subgroups with EMM that align with known risk factors in the existing literature, confirming the ability of our method to identify risk groups of AF successfully. In addition, new hypotheses on found characteristics and combinations thereof have originated from the deployment. The hospital is advised to administer preventive medications to patients who match the descriptions of the risk groups found and perform follow-up clinical studies to validate the found hypotheses.
The chemical industry is faced with the urgent challenge of effectively harnessing the vast amounts of time-series data generated by thousands of sensors, which is essential for forecasting chemical states, achieving accurate real-time control of production processes. Traditional forecasting methods suffer from high computational latency and struggle with the complexity of spatiotemporal dependencies. As a result, modeling this data becomes challenging. This paper introduces a novel approach, referred to as ASTNet, designed to address these challenges. ASTNet integrates an asynchronous spatiotemporal modeling framework that combines temporal and spatial encoders, enabling concurrent learning of temporal and spatial dependencies while reducing computational latency. Additionally, it introduces a gated graph fusion mechanism that adaptively combines static (meta) and evolving (dynamic) sensor graphs, enhancing the handling of heterogeneous sensor data and spatial correlations. Extensive experiments on three real-world chemical sensor datasets demonstrate that ASTNet outperforms SOTA methods in terms of both prediction accuracy and computational efficiency, making ASTNet successfully deployed in chemical engineering industrial scenarios.
Scientific computing for large deformation of elastic-plastic solids is critical for numerous real-world applications. Classical numerical solvers rely primarily on local discrete linear approximation and are constrained by an inherent trade-off between accuracy and efficiency. Recently, deep learning models have achieved impressive progress in solving the continuum mechanism. While previous models have explored various architectures and constructed coefficient-solution mappings, they are designed for general instances without considering specific problem properties and hard to accurately handle with complex elastic-plastic solids involving contact, loading and unloading. In this work, we take stretch bending, a popular metal fabrication technique, as our case study and introduce LaDEEP, a deep learning-based surrogate model for La rge De formation of Elastic-Plastic Solids. We encode the partitioned regions of the involved slender solids into a token sequence to maintain their essential order property. To characterize the physical process of the solid deformation, a two-stage Transformer-based module is designed to predict the deformation with the sequence of tokens as input. Empirically, LaDEEP achieves five magnitudes faster speed than finite element methods with a comparable accuracy, and gains 20.47% relative improvement on average compared to other deep learning baselines. We have also deployed our model into a real-world industrial production system, and it has shown remarkable performance in both accuracy and efficiency. Code is available at https://github.com/therontau0054/LaDEEP.
Retrieval Augmented Cross-Domain LifeLong Behavior Modeling for Enhancing Click-through Rate Prediction
PDF ↗Lifelong behavior modeling for single-domain has been widely investigated in industry click-through (CTR) prediction. However, some domains do not always have rich historical behaviors in online platforms, so cross-domain lifelong behavior modeling is overlooked. This paper proposes a novel retrieval augmented lifelong cross-domain net (RAL-CDNet) to address the challenges in cross-domain lifelong behavior modeling. There are three components in RAL-CDNet, i.e., cross-domain retrieval unit, cross-domain alignment unit, and cross-net. As the general search unit in the previous study, a cross-domain retrieval unit features a retrieval augmented paradigm that utilizes a pre-trained language model to learn the intrinsic textual information of user behaviors and generates the sequential behaviors from the source domain based on sequential behaviors in the target domain. The retrieval augmented behaviors can achieve consistency and capture accurate hidden interest for target domain CTR prediction. Furthermore, we propose the cross-domain alignment unit to align the embeddings across domains by adding a semantic-guided contrastive loss and auxiliary task loss in the source domain. This allows the embeddings to be consistent across domains and have enough source information to capture the cross-domain relation. Finally, the cross-net utilizes two-level attention techniques to enhance the final prediction in the target domain. We conduct extensive experiments on both a public dataset and an industrial dataset from the WeChat advertising platform to demonstrate the effectiveness of RAL-CDNet in terms of offline and online metrics.
Tagging systems play an essential role in various information retrieval applications such as search engines and recommender systems. Recently, Large Language Models (LLMs) have been applied in tagging systems due to their extensive world knowledge, semantic understanding, and reasoning capabilities. Despite achieving remarkable performance, existing methods still have limitations, including difficulties in retrieving relevant candidate tags comprehensively, challenges in adapting to emerging domain-specific knowledge, and the lack of reliable tag confidence quantification. To address these three limitations above, we propose an automatic tagging system LLM4Tag. First, a graph-based tag recall module is designed to effectively and comprehensively construct a small-scale highly relevant candidate tag set. Subsequently, a knowledge-enhanced tag generation module is employed to generate accurate tags with long-term and short-term knowledge injection. Finally, a tag confidence calibration module is introduced to generate reliable tag confidence scores. Extensive experiments over three large-scale industrial datasets show that LLM4Tag significantly outperforms the state-of-the-art baselines and LLM4Tag has been deployed online for content tagging to serve hundreds of millions of users.
Audio-Enhanced Vision-Language Modeling with Latent Space Broadening for High Quality Data Expansion
Transformer-based multimodal models are widely used in industrial-scale recommendation, search, and advertising systems for content understanding and relevance ranking. Enhancing labeled training data quality and cross-modal fusion significantly improves model performance, influencing key metrics such as quality view rates and ad revenue. High-quality annotations are crucial for advancing content modeling, yet traditional statistical-based active learning (AL) methods face limitations: they struggle to detect overconfident misclassifications and are less effective in distinguishing semantically similar items in deep neural networks. Additionally, audio information plays an increasing role, especially in short-video platforms, yet most pretrained multimodal architectures primarily focus on text and images. While training from scratch across all three modalities is possible, it sacrifices the benefits of leveraging existing pretrained visual-language (VL) and audio models. To address these challenges, we propose kNN-based Latent Space Broadening (LSB) to enhance AL efficiency, achieving an up to 9% recall improvement at 80% precision on proprietary datasets. Additionally, we introduce Vision-Language Modeling with Audio Enhancement (VLMAE), a mid-fusion approach integrating audio into VL models, yielding up to another 9% recall improvement at 80% precision. Our methods are successfully deployed in multiple production systems, leading to significant business gains through online A/B experiments.
Enhancing Bone Mineral Density Estimation from X-ray Images with Cross-Modal Knowledge Distillation
PDF ↗Dual-energy X-ray absorptiometry (DXA) enables accurate bone mineral density but requires specialized equipment and protocols. X-ray-based BMD screening offers opportunistic early detection, though prior methods struggle with X-ray intensity variations and demand large datasets. We introduce a cross-modal knowledge distillation BMD prediction framework (CMKD-BMD) fusing X-ray/CT data to enhance X-ray-only BMD prediction. Each single-modal student network employs a multi-scale visual extractor for hierarchical features, an unsupervised graph-based structural learner for anatomical relationships, and an adaptive fusion module to generate a unified representation. The teacher network integrates single-modal representations from students and transfers multimodal knowledge from the teacher to students. Our model outperforms existing methods on both the collected dataset (comprising 1620 X-rays and 280 CT cases) and the publicly available VerSe2019 dataset, demonstrating superior BMD estimation performance. Furthermore, we developed OrthoSim, an orthopedic surgical simulation platform with CMKD-BMD, which has shown promising clinical effectiveness in trial evaluations. Our code is available at https://github.com/KeyueShi/CMKD-BMD.
xisting online advertising systems generate high-quality ad recommendations through a complex online serving pipeline whenever a user's ad request arrives. However, our analyses on the display advertising system in Taobao show that this paradigm could lead to an inefficient utilization of computational resources. In this work, we propose an intelligent computation recycling framework called ComRecycle, which caches and reuses unexposed ad sets for repetitive ad requests to improve the computational resource utilization. We introduce fine-grained computation recycling strategies, and formulate the computation recycling decision as an online constrained optimization problem. Therefore, ComRecycle can achieve the goal of reducing computation costs while guaranteeing the same level of recommendation performance. Extensive offline experiments validate the correctness and effectiveness of ComRecycle. Our online A/B testing demonstrates that ComRecycle can save over 20% computational resources while maintaining the same system performance as the baseline.
Incident management remains a critical yet challenging task for large-scale cloud services. Most cloud service providers abstract troubleshooting into predefined workflows for different incidents, offering step-by-step guidance. However, manually crafting workflows is resource-consuming and knowledge-intensive, hindering large-scale deployment. Most automated techniques for workflow orchestration rely on large language models (LLMs) to handle complex tasks but overlook key aspects of troubleshooting, including complex expertise, domain requirements, and the reliability of AI feedback. These limitations undermine workflow quality. Therefore, we propose FlowXpert, a novel framework for troubleshooting workflow orchestration. Leveraging LLMs, it first builds a knowledge base centered on incident-aware nodes to precisely depict expertise. Then, fed into AI feedback and synthetic preference data, reinforcement learning is applied to refine the workflow generator and evaluator. To assess troubleshooting workflows, we introduce OpsFlowBench based on Huawei Cloud's datacenter switch operation documents. Benchmark tests under the tailored STEPScore metric validate its effectiveness. Furthermore, during a 10-week deployment in Huawei Cloud's datacenter network, FlowXpert provided valuable support to both on-call engineers and AI executors, as evidenced by empirical data and case study.
Maintaining a healthy ecosystem in billion-scale online platforms is challenging, as users naturally gravitate toward popular items, leaving cold and less-explored items behind. This ''rich-get-richer'' phenomenon hinders the growth of potentially valuable cold items and harms the platform's ecosystem. Existing cold-start models primarily focus on improving initial recommendation performance for cold items but fail to address users' natural preference for popular content. In this paper, we introduce AliBoost, Alibaba's ecological boosting framework, designed to complement user-oriented natural recommendations and foster a healthier ecosystem. AliBoost incorporates a tiered boosting structure and boosting principles to ensure high-potential items quickly gain exposure while minimizing disruption to low-potential items. To achieve this, we propose the Stacking Fine-Tuning Cold Predictor to enhance the foundation CTR model's performance on cold items for accurate CTR and potential prediction. AliBoost then employs an Item-oriented Bidding Boosting mechanism to deliver cold items to the most suitable users while balancing boosting speed with user-personalized preferences. Over the past six months, AliBoost has been deployed across Alibaba's mainstream platforms, successfully cold-starting over a billion new items and increasing both clicks and GMV of cold items by over 60% within 180 days. Extensive online analysis and A/B testing demonstrate the effectiveness of AliBoost in addressing ecological challenges, offering new insights into the design of billion-scale recommender systems.
GREAT: Guiding Query Generation with a Trie for Recommending Related Search about Video at Kuaishou
PDF ↗Currently, short video platforms have become the primary place for individuals to share experiences and obtain information. To better meet users' needs for acquiring information while browsing short videos, some apps have introduced a search entry at the bottom of videos, accompanied with recommended relevant queries. This scenario is known as query recommendation in video-related search, where core task is item-to-query (I2Q) recommendation. As this scenario has only emerged in recent years, there is a notable scarcity of academic research and publicly available datasets in this domain. To address this gap, we systematically examine the challenges associated with this scenario for the first time. Subsequently, we release a large-scale dataset derived from real-world data pertaining to the query recommendation in video- related search on the Kuaishou app (KuaiRS). Presently, existing methods rely on embeddings to calculate similarity for matching short videos with queries, lacking deep interaction between the semantic content and the query. In this paper, we introduce a novel LLM-based framework named GREAT, which guides que ry g ener ation with a trie to address I2Q recommendation in related search. Specifically, we initially gather high-quality queries with high exposure and click-through rate to construct a query-based trie. During training, we enhance the LLM's capability to generate high-quality queries using the query-based trie. In the inference phase, the query-based trie serves as a guide for the token generation. Finally, we further refine the relevance and literal quality between items and queries via a post-processing module. Extensive offline and online experiments demonstrate the effectiveness of our proposed method.
Graph neural networks (GNNs) have emerged as state-of-the-art methods to learn from graph-structured data for recommendation. However, most existing GNN-based recommendation methods focus on the optimization of model structures and learning strategies based on pre-defined graphs, neglecting the importance of the graph construction stage. Earlier works for graph construction usually rely on specific rules or crowdsourcing, which are either too simplistic or too labor-intensive. Recent works start to utilize large language models (LLMs) to automate the graph construction, in view of their abundant open-world knowledge and remarkable reasoning capabilities. Nevertheless, they generally suffer from two limitations: (1) invisibility of global view (e.g., overlooking contextual information) and (2) construction inefficiency. To this end, we introduce AutoGraph, an automatic graph construction framework based on LLMs for recommendation. Specifically, we first use LLMs to infer the user preference and item knowledge, which is encoded as semantic vectors. Next, we employ vector quantization to extract the latent factors from the semantic vectors. The latent factors are then incorporated as extra nodes to link the user/item nodes, resulting in a graph with in-depth global-view semantics. We further design metapath-based message aggregation to effectively aggregate the semantic and collaborative information. The framework is model-agnostic and compatible with different backbone models. Extensive experiments on three real-world datasets demonstrate the efficacy and efficiency of AutoGraph compared to existing baseline methods. We have deployed AutoGraph in Huawei advertising platform, and gain a 2.69% improvement on RPM and a 7.31% improvement on eCPM in the online A/B test. Currently AutoGraph has been used as the main traffic model, serving hundreds of millions of people.
Illegal logging poses a significant threat to global biodiversity, climate stability, and depresses international prices for legal wood harvesting and responsible forest products trade, affecting livelihoods and communities across the globe. Stable isotope ratio analysis (SIRA) is rapidly becoming an important tool for determining the harvest location of traded, organic, products. The spatial pattern in stable isotope ratio values depends on factors such as atmospheric and environmental conditions and can thus be used for geographic origin identification. We present here the results of a deployed machine learning pipeline where we leverage both isotope values and atmospheric variables to determine timber harvest location. Additionally, the pipeline incorporates uncertainty estimation to facilitate the interpretation of harvest location determination for analysts. We present our experiments on a collection of oak (Quercus spp.) tree samples from its global range. Our pipeline outperforms comparable state-of-the-art models determining geographic harvest origin of commercially traded wood products, and has been used by European enforcement agencies to identify harvest location misrepresentation. We also identify opportunities for further advancement of our framework and how it can be generalized to help identify the origin of falsely labeled organic products throughout the supply chain.
In geoscience, it is necessary to study the lithography of the Earth's subsurface, which consists of different stratified layers called geological formations. This study performs well correlation task to model and characterize reservoirs. This operation links the beginning of specific geological formations called tops using measurements from drilled wells. Although data are abundant, the traditional algorithms used for well correlation are semi-automated, requiring significant time and high computational cost. This paper introduces GeoTS, a Python library to apply cutting-edge time series classification models to perform well correlation in a completely automated setting. As input, take the drilling trajectory depth and gamma-ray well logs, which measure the natural radioactivity across the well depth trajectory. The top depths of the formations are predicted as an output. The gamma-ray signatures are extracted around the top depths assigned by geologists. Preprocessing is performed to clean and cluster these signatures using the Dynamic Time Wrapping (DTW) distance and HDBSCAN. Implementation of existing deep learning architectures (FCN, InceptionTime, XceptionTime, XCM, LSTM-FCN) and new architecture (LSTM-2dCNN, LSTM-XCM) are performed. Our experiments demonstrate faster computation with an increase in accuracy. GradCAM has also been implemented for model explainability. Experiments were performed using Colorado oil fields and deployed on Wyoming oil fields. The deployment has provided us with critical insights regarding the improvements needed.
Millions of smart meters have been deployed worldwide, collecting the total power consumed by individual households. Based on these data, electricity suppliers offer their clients energy monitoring solutions to provide feedback on the consumption of their individual appliances. Historically, such estimates have relied on statistical methods that use coarse-grained total monthly consumption and static customer data, such as appliance ownership. Non-Intrusive Load Monitoring (NILM) is the problem of disaggregating a household's collected total power consumption to retrieve the consumed power for individual appliances. Current state-of-the-art (SotA) solutions for NILM are based on deep-learning (DL) and operate on subsequences of an entire household consumption reading. However, the non-stationary nature of real-world smart meter data leads to a drift in the data distribution within each segmented window, which significantly affects model performance. This paper introduces NILMFormer, a Transformer-based architecture that incorporates a new subsequence stationarization/de-stationarization scheme to mitigate the distribution drift and that uses a novel positional encoding that relies only on the subsequence's timestamp information. Experiments with 4 real-world datasets show that NILMFormer significantly outperforms the SotA approaches. Our solution has been deployed as the backbone algorithm for EDF's (Electricité De France) consumption monitoring service, delivering detailed insights to millions of customers about their individual appliances' power consumption.