Symbolic Regression (SR) algorithms are powerful machine learning tools that discover mathematical expressions from data, but their evaluation is often hindered by the computational cost of optimizing numerical parameters (also called constants) within candidate expressions. To address this limitation, we introduce POVE (PreOptimized Vault of Expressions), a comprehensive repository of preoptimized symbolic expressions where numerical parameters have been efficiently computed using an established optimization technique in SR (i.e., BFGS) on widely used SR regression problems (i.e., SRBench). In addition to optimized constants, POVE stores the corresponding fitness values, quantifying the accuracy of each expression on its respective regression problem, allowing researchers to directly retrieve both optimized expressions and their precomputed performance metrics from memory (e.g., hash tables or dictionaries). By eliminating the need for computationally expensive numerical optimization steps, POVE enables significantly faster evaluation, large-scale benchmarking, and extensive hyperparameter analysis. It provides a structured and standardized foundation for comparing SR algorithms, assessing their consistency across large numbers of random seeds and investigating the impact of hyperparameters without the overhead of numerical parameter optimization. By offering a collection of precomputed expressions, parameters, and fitness values across diverse regression problems, POVE enhances reproducibility, accelerates experimental workflows, and enables scalable SR research. It serves as a valuable resource for both developing novel SR methodologies and improving existing algorithms by providing a reliable reference for optimal expression discovery. With POVE, SR research can shift away from costly optimization, enabling deeper algorithmic exploration and more comprehensive systematic evaluation. POVE also lowers the barrier to entry for SR research and makes large-scale evaluation significantly more accessible. POVE is publicly available at: https://github.com/kentridgeai/POVE
论文检索
输入标题、作者或关键词，从 2,072 篇学术成果中精准定位
Robust causal discovery in time series datasets depends on reliable benchmark datasets with known ground-truth causal relationships. However, such datasets remain scarce, and existing synthetic alternatives often overlook critical temporal properties inherent in real-world data, including nonstationarity driven by trends and seasonality, irregular sampling intervals, and the presence of unobserved confounders. To address these challenges, we introduce TimeGraph, a comprehensive suite of synthetic time-series benchmark datasets that systematically incorporates both linear and nonlinear dependencies while modeling key temporal characteristics such as trends, seasonal effects, and heterogeneous noise patterns. Each dataset is accompanied by a fully specified causal graph featuring varying densities and diverse noise distributions and is provided in two versions: one including unobserved confounders and one without, thereby offering extensive coverage of real-world complexity while preserving methodological neutrality. We further demonstrate the utility of TimeGraph through systematic evaluations of state-of-the-art causal discovery algorithms including PCMCI+, LPCMCI, and FGES across a diverse array of configurations and metrics. Our experiments reveal significant variations in algorithmic performance under realistic temporal conditions, underscoring the need for robust synthetic benchmarks in the fair and transparent assessment of causal discovery methods. The complete TimeGraph suite, including dataset generation scripts, evaluation metrics, and recommended experimental protocols, is freely available to facilitate reproducible research and foster community-driven advancements in time-series causal discovery.
As large language models (LLMs) continue to advance and gain widespread use, establishing systematic and reliable evaluation methodologies for LLMs and vision-language models (VLMs) has become essential to ensure their real-world effectiveness and reliability. There have been some early explorations about the usability of LLMs for limited urban tasks, but a systematic and scalable evaluation benchmark is still lacking. The challenge in constructing a systematic evaluation benchmark for urban research lies in the diversity of urban data, the complexity of application scenarios and the highly dynamic nature of the urban environment. In this paper, we design CityBench, an interactive simulator based evaluation platform, as the first systematic benchmark for evaluating the capabilities of LLMs for diverse tasks in urban research. First, we build CityData to integrate the diverse urban data and CitySimu to simulate fine-grained urban dynamics. Based on CityData and CitySimu, we design 8 representative urban tasks in 2 categories of perception-understanding and decision-making as the CityBench. With extensive results from 30 well-known LLMs and VLMs in 13 cities around the world, we find that advanced LLMs and VLMs can achieve competitive performance in diverse urban tasks requiring commonsense and semantic understanding abilities, e.g., understanding the human dynamics and semantic inference of urban images. Meanwhile, they fail to solve the challenging urban tasks requiring professional knowledge and high-level numerical abilities, e.g., geospatial prediction and traffic control task. These findings provide critical insights for the effective utilization and further development of LLMs to advance urban-related tasks and research in the future.
The analysis of the privacy properties of Privacy-Preserving Ads APIs is an area of research that has received strong interest from academics, industry, and regulators. Despite this interest, the empirical study of these methods is severely hindered by the lack of publicly available data. Reliable empirical analysis of the privacy properties of an API, in fact, requires access to a dataset consisting of realistic API outputs for a large collection of users; however, privacy concerns prevent the general release of such data to the public. In this work, we address this problem by developing a novel methodology to construct synthetic API outputs that are simultaneously realistic enough to enable accurate study and provide strong privacy protections. We focus on one of the Privacy-Preserving Ads APIs: the Topics API, part of Google Chrome's Privacy Sandbox, which enables interest-based advertising without relying on third-party cookies. We developed a methodology to generate a differentially-private dataset that closely matches the re-identification risk properties of the real Topics API data. The use of differential privacy provides strong theoretical bounds on the leakage of private user information from this release. Our methodology is based on first computing a large number of differentially-private statistics describing how output API traces evolve over time. Then, we design a parameterized distribution over sequences of API traces and optimize its parameters so that they closely match the statistics obtained. Finally, we create the synthetic data by drawing from this distribution. Our work is complemented by an open-source release of the anonymized dataset obtained by this methodology. We hope this will enable external researchers to analyze the API in-depth and replicate prior and future work on a realistic large-scale dataset. We believe that this work will contribute to fostering transparency regarding the privacy properties of Privacy-Preserving Ads APIs.
The rapid rise of online social networks underscores the need to understand the heterogeneous strengths of online relationships. Yet, efforts to assess tie strength (TS) are hindered by the lack of ground-truth labels, differing research perspectives, and limited model performance in real-world settings. To address this gap, we introduce BTS, a comprehensive Benchmark for Tie Strength prediction, aiming to establish a standardized foundation for evaluating and advancing TS prediction methodologies. Specifically, our contributions are: TS Pseudo-Label Techniques -- we categorize TS into seven standardized pseudo-labeling techniques based on prior literature; TS Dataset Collection -- we present a representative collection of three social networks and perform data analysis by investigating the class distributions and correlations across the generated pseudo-labels; TS Pseudo-Label Evaluation Framework -- we propose a standardized framework to evaluate the pseudo-label quality from the perspective of tie resilience; Benchmarking -- we evaluate existing tie strength prediction model performance using the BTS dataset collection, exploring the effects of different experiment settings, models, and evaluation criteria on the results. Furthermore, we derive key insights to enhance existing methods and shed light on promising directions for future research in this domain. The BTS dataset collection, along with the curation codes and experimental scripts, is all available at: https://github.com/XueqiC/Awesome-Tie-Strength-Prediction.
Metamaterials, engineered materials with architected structures across multiple length scales, offer unprecedented and tunable mechanical properties that surpass those of conventional materials. However, leveraging advanced machine learning (ML) for metamaterial discovery is hindered by three fundamental challenges: (C1) Data Heterogeneity Challenge arises from heterogeneous data sources, heterogeneous composition scales, and heterogeneous structure categories; (C2) Model Complexity Challenge stems from the intricate geometric constraints of ML models, which complicate their adaptation to metamaterial structures; and (C3) Human-AI Collaboration Challenge comes from the “dual black-box” nature of sophisticated ML models and the need for intuitive user interfaces. To tackle these challenges, we introduce a unified framework, named MetamatBench, that operates on three levels. (1) At the data level, we integrate and standardize 5 heterogeneous, multi-modal metamaterial datasets. (2) The ML level provides a comprehensive toolkit that adapts 17 state-of-the-art ML methods for metamaterial discovery. It also includes a comprehensive evaluation suite with 12 novel performance metrics with finite element-based assessments to ensure accurate and reliable model validation. (3) The user level features a visual-interactive interface that bridges the gap between complex ML techniques and non-ML researchers, advancing property prediction and inverse design of metamaterials for research and applications. MetamatBench offers a unified platform that enables machine learning researchers and practitioners to develop and evaluate new methodologies in metamaterial discovery. For accessibility and reproducibility, we open-source our benchmark and the code-base at https://github.com/cjpcool/Metamaterial-Benchmark.
In this paper, we introduce the Generalized Auto Segmentation (GAS), an end-to-end, cross-domain personalization platform developed at Meta. GAS is designed to optimize personalization policies for online experiments, operating independently of any specific server infrastructure. Our process begins with a seed experiment to collect user data, which is followed by feature logging and selection. User segmentation is then performed using a feature-threshold method. Specifically, we utilize simple combinations of single or dual features to identify concise, explainable, and sustainable user cohorts that enhance both business outcomes and user experience over the long term. Treatments and the reward function for these segments are refined through ParallAX, a novel methodology at Meta that uses offline estimates to optimize online metrics, ensuring impactful and precisely tailored interventions for diverse user groups. The deployment of GAS across various Meta productions has demonstrated its efficiency and effectiveness.
Industry-grade ML models are carefully designed to meet rapidly evolving serving constraints, which requires significant resources for model development. In this paper, we propose MatTA, a framework for training multiple accurate Student models using a novel Teacher-TA-Student recipe. TA models are larger versions of the Student models with higher capacity, and thus allow Student models to better relate to the Teacher model and also bring in more domain-specific expertise. Furthermore, multiple accurate Student models can be extracted from the TA model. Therefore, despite only one training run, our methodology provides multiple servable options to trade off accuracy for lower serving cost. We demonstrate the proposed method, MatTA, on proprietary datasets and models. Its practical efficacy is underscored by live A/B tests within a production ML system, demonstrating 20% improvement on a key metric. We also demonstrate our method on GPT-2 Medium, a public model, and achieve relative improvements of over 24% on SAT Math and over 10% on the LAMBADA benchmark.
LinkedIn, one of the world's largest platforms for professional networking and job seeking, encounters various modeling challenges in building recommendation systems for its job matching product, including cold-start, filter bubbles, and biases affecting candidate-job matching. To address these, we developed the STAR (Signal integration for Talent And Recruiters) system, leveraging the combined strengths of Large Language Models (LLMs) and Graph Neural Networks (GNNs). LLMs excel at understanding textual data, such as member profiles and job postings, while GNNs capture intricate relationships and mitigate cold-start issues through network effects. STAR integrates diverse signals by uniting LLM and GNN capabilities with industrial-scale paradigms including adaptive sampling and version management. It provides an end-to-end solution for developing and deploying embeddings in large-scale recommender systems. Our key contributions include a robust methodology for building embeddings in industrial applications, a scalable GNN-LLM integration for high-performing recommendations, and practical insights for real-world model deployment.
Synergizing Implicit and Explicit User Interests: A Multi-Embedding Retrieval Framework at Pinterest
PDF ↗Industrial recommendation systems are typically composed of multiple stages, including retrieval, ranking, and blending. The retrieval stage plays a critical role in generating a high-recall set of candidate items that covers a wide range of diverse user interests. Effectively covering the diverse and long-tail user interests within this stage poses a significant challenge: traditional two-tower models struggle in this regard due to limited user-item feature interaction and often bias towards top use cases. To address these issues, we propose a novel multi-embedding retrieval framework designed to enhance user interest representation by generating multiple user embeddings conditioned on both implicit and explicit user interests. Implicit interests are captured from user history through a Differentiable Clustering Module (DCM), whereas explicit interests, such as topics that the user has followed, are modeled via Conditional Retrieval (CR). These methodologies represent a form of conditioned user representation learning that involves condition representation construction and associating the target item with the relevant conditions. Synergizing implicit and explicit user interests serves as a complementary approach to achieve more effective and comprehensive candidate retrieval as they benefit on different user segments and extract conditions from different but supplementary sources. Extensive experiments and A/B testing reveal significant improvements in user engagements and feed diversity metrics. Our proposed framework has been successfully deployed on Pinterest home feed.
The proliferation of complex non-factoid questions in modern information seeking (IS) systems exposes critical limitations in conventional Retrieval-Augmented Generation (RAG) approaches, particularly their static search strategies and the lack of systematic multi-source information integration capabilities. Facing these limitations, we present PASS (Proactive Agent-driven Search System), a novel multi-agent framework that operationalizes human-like proactive search strategies through five specialized agents: Revealer for intent analysis, Navigator for search planning, Seeker/Reader for adaptive retrieval, and Writer for response synthesis, systematically expanding the search space through iterative query refinement and multi-perspective knowledge integration. Crucially, our framework demonstrates remarkable adaptability to mid-sized LLMs, demonstrating its scalability in resource-constrained environments. To comprehensively assess the effectiveness of the proposed framework, we carry out extensive experiments on both mid-sized and proprietary large-scale LLMs, evaluating response quality for complex non-factoid questions using a newly introduced nugget-based assessment. Experimental results from offline nugget-based evaluation and online A/B Tests confirm substantial improvements in answer quality, advancing proactive information seeking methodologies and offering practical pathways for democratizing complex reasoning capabilities to resource-constrained environments.
With the growing availability of large data repositories within and across organizations, it is becoming feasible to selectively acquire data in the wild for data augmentation, tailored to specific downstream tasks. However, current methodologies concentrate primarily on single-dimensional augmentation tasks for tabular data, such as increasing the number of data points or enriching features. These approaches are not designed for trajectory data, which cannot fully utilize its spatio-temporal characteristics, resulting in suboptimal performance. To address this gap, we present Toast, an iterative framework of task-oriented multi-dimensional augmentation for spatio-temporal trajectory data. Toast supports all three augmentation operations and consists of three core components: (1) augmentation candidate search, which identifies unionable or joinable augmentation candidates in the wild; (2) augmentation candidate selection, which defines and selects the ''best'' augmentations that offer the greatest benefit to downstream tasks; (3) augmentation type selection, which determines the most cost-effective type of augmentation to perform based on a cost model. Extensive experiments on four classical trajectory data mining applications and datasets demonstrate that Toast can enhance model performance by 5%-15% and achieve more than a 2× improvement in efficiency.
Context Pooling: Query-specific Graph Pooling for Generic Inductive Link Prediction in Knowledge Graphs
PDF ↗Recent investigations on the effectiveness of Graph Neural Network (GNN)-based models for link prediction in Knowledge Graphs (KGs) show that vanilla aggregation does not significantly impact the model performance. In this paper, we introduce a novel method, named Context Pooling, to enhance GNN-based models' efficacy for link predictions in KGs. To our best of knowledge, Context Pooling is the first methodology that applies graph pooling in KGs. Additionally, Context Pooling is first-of-its-kind to enable the generation of query-specific graphs for inductive settings, where testing entities are unseen during training. Specifically, we devise two metrics, namely neighborhood precision and neighborhood recall, to assess the neighbors' logical relevance regarding the given queries, thereby enabling the subsequent comprehensive identification of only the logically relevant neighbors for link prediction. Our method is generic and assessed by being applied to two state-of-the-art (SOTA) models on three public transductive and inductive datasets, achieving SOTA performance in 42 out of 48 settings.
Distribution shifts between training and testing datasets can significantly affect the effectiveness of machine learning models, as the typical assumption of data homogeneity often does not hold in practical applications. To address this challenge, previous literature explores invariant representation learning methods, including but not limited to domain adversarial learning, feature alignment, and invariant risk minimization. In this paper, we introduce a novel method for generating distribution-free prediction regions, which effectively quantify the uncertainty of predictions in different environments. Our methodology incorporates a weighted conformity score, tailored to the environment of each test sample, to construct adaptive conformal intervals. The proposed method is computationally efficient and can be implemented by subsampling. We demonstrate how these intervals maintain a consistent conditional coverage under some conditions. The numerical experiments, including simulations and real data analysis, show that our method is applicable and robust in dealing with distribution shifts.
Multimodal image fusion and semantic segmentation are critical for autonomous driving. Despite advancements, current models often struggle with segmenting densely packed elements due to a lack of comprehensive fusion features for guidance during training. While the Segment Anything Model (SAM) allows precise control during fine-tuning through its flexible prompting encoder, its potential remains largely unexplored in the context of multimodal segmentation for natural images. In this paper, we introduce SAM into multimodal image segmentation for the first time, proposing a novel framework that combines Latent Space Token Generation (LSTG) and Fusion Mask Prompting (FMP) modules. This approach transforms the training methodology for multimodal segmentation from a traditional black-box approach to a controllable, prompt-based mechanism. Specifically, we obtain latent space features for both modalities through vector quantization and embed them into a cross-attention-based inter-domain fusion module to establish long-range dependencies between modalities. We then use these comprehensive fusion features as prompts to guide precise pixel-level segmentation. Extensive experiments on multiple public datasets demonstrate that our method significantly outperforms SAM and SAM2 in multimodal autonomous driving scenarios, achieving an average improvement of 4.1% over the state-of-the-art method in segmentation mIoU, and the performance is also optimized in other multi-modal visual scenes.
Classical time-series forecasting methods rely on learning from a fixed context window of past observations and extrapolating to a fixed number of future steps. They are therefore limited in flexibility and effectiveness in modeling tasks of varying historical lengths or future horizons, requiring re-training for each setting. We propose Contextual Meta-Adaptation (CMA), a unified scheme that captures history contexts and extrapolates future horizons with a single training. CMA integrates denoising diffusion capability which can sequentially generate a future series with maximum likelihood and remains consistent with observed history and past generations. We develop an efficient update strategy using gradient-based meta-learning for full-parameter tuning during training and LoRA adaptation at testing time. We validate our CMA scheme on in-context learning with longer observed history, ex(tend)-context learning with up to 3 times longer future horizons, and transfer-learning across different source-target domains. Our CMA outperforms existing methods by an average of 7% on six academic datasets and a significant 16% improvement on four realistic marketing datasets.
Probabilistic time series imputation has been widely applied in real-world scenarios due to its ability for uncertainty estimation and denoising diffusion probabilistic models (DDPMs) have achieved great success in probabilistic time series imputation tasks with its power to model complex distributions. However, current DDPM-based probabilistic time series imputation methodologies are confronted with two types of challenges: 1) The backbone modules of the denoising parts are not capable of achieving sequence modeling with low time complexity. 2) The architecture of denoising modules can not handle the dependencies in the time series data effectively. To address the first challenge, we explore the potential of state space model, namely Mamba, as the backbone denoising module for DDPMs. To tackle the second challenge, we carefully devise several SSM-based blocks for time series data modeling. Experimental results demonstrate that our approach can achieve state-of-the-art time series imputation results on multiple real-world datasets. Our datasets and code are available at https://github.com/decisionintelligence/SSD-TS/
Decoupling Spatio-Temporal Prediction: When Lightweight Large Models Meet Adaptive Hypergraphs
PDF ↗Spatio-temporal prediction is a pivotal task with broad applications in traffic management, climate monitoring, energy scheduling, etc. However, existing methodologies often struggle to balance model expressiveness and computational efficiency, especially when scaling to large real-world datasets. To tackle these challenges, we propose STH-SepNet (Spatio-Temporal Hypergraph Separation Networks), a novel framework that decouples temporal and spatial modeling to enhance both efficiency and precision. Therein, the temporal dimension is modeled using lightweight large language models, which effectively capture low-rank temporal dynamics. Concurrently, the spatial dimension is addressed through an adaptive hypergraph neural network, which dynamically constructs hyperedges to model intricate, higher-order interactions. A carefully designed gating mechanism is integrated to seamlessly fuse temporal and spatial representations. By leveraging the fundamental principles of low-rank temporal dynamics and spatial interactions, STH-SepNet offers a pragmatic and scalable solution for spatio-temporal prediction in real-world applications. Extensive experiments on large-scale real-world datasets across multiple benchmarks demonstrate the effectiveness of STH-SepNet in boosting predictive performance while maintaining computational efficiency. This work may provide a promising lightweight framework for spatio-temporal prediction, aiming to reduce computational demands and while enhancing predictive performance. Our code is avaliable at https://github.com/SEU-WENJIA/ST-SepNet-Lightweight-LLMs-Meet-Adaptive-Hypergraphs.
Applying large language models (LLMs) to academic API usage shows promise in reducing researchers' efforts to seek academic information. However, current LLM methods for using APIs struggle with the complex API coupling commonly encountered in academic queries. To address this, we introduce SoAy, a solution-based LLM methodology for academic information seeking. SoAy enables LLMs to generate code for invoking APIs, guided by a pre-constructed API calling sequence referred to as a solution. This solution simplifies the model's understanding of complex API relationships, while the generated code enhances reasoning efficiency. LLMs are aligned with this solution-oriented, code-based reasoning method by automatically enumerating valid API coupling sequences and transforming them into queries and executable code. To evaluate SoAy, we introduce SoAyBench, an evaluation benchmark accompanied by SoAyEval, built upon a cloned environment of APIs from AMiner. Experimental results demonstrate a 34.58-75.99% performance improvement compared to state-of-the-art LLM API-based baselines. All datasets, codes, tuned models, and deployed online services are publicly accessible at https://github.com/RUCKBReasoning/SoAy.
We present LinkSAGE, an innovative framework that integrates Graph Neural Networks (GNNs) into large-scale personalized job matching systems, designed to address the complex dynamics of LinkedIn's extensive professional network. Our approach capitalizes on a novel job marketplace graph, the largest and most intricate of its kind in industry, with billions of nodes and edges. This graph is not merely extensive but also richly detailed, encompassing member and job nodes along with key attributes, thus creating an expansive and interwoven network. A key innovation in LinkSAGE is its training and serving methodology, which effectively combines inductive graph learning on a heterogeneous, evolving graph with an encoder-decoder GNN model. This methodology decouples the training of the GNN model from that of existing Deep Neural Network (DNN) models, eliminating the need for frequent GNN retraining while maintaining up-to-date graph signals in near real-time, allowing for the effective integration of GNN insights through transfer learning. The subsequent nearline inference system serves the GNN encoder within a real-world setting, significantly reducing online latency and obviating the need for costly real-time GNN infrastructure. Validated across multiple online A/B tests in diverse product scenarios, LinkSAGE demonstrates marked improvements in member engagement, relevance matching, and member retention, confirming its generalizability and practical impact.