Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective
UC San Diego · University of California, San Diego · Shanghai Jiaotong University · ucsd · Purdue University · CMU, Carnegie Mellon University · University of California, San Diego || MBZUAI Institute of Foundation Models · Petuum Inc. · University of Cambridge · Mohamed bin Zayed University of Artificial Intelligence · MBZUAI Institute of Foundation Models · CMU/MBZUAI/GenBio · xAI / UCSD
PDF 由论文原始站点提供，PaperCompass 不保存论文文件。
摘要
Reinforcement learning (RL) has shown promise in enhancing large language model (LLM) reasoning, yet progress towards broader capabilities is limited by the availability of high-quality, multi-domain datasets. This work introduces \ours, a 92K RL-for-reasoning dataset designed to address this gap, covering six reasoning domains: Math, Code, Science, Logic, Simulation, and Tabular, each with corresponding verifiers. We build \ours via a careful data-curation pipeline, including sourcing, deduplication, reward design, and domain-specific and difficulty-based filtering, to facilitate the systematic investigation of cross-domain RL generalization. Our study using \ours suggests the efficacy of a simple mixed-domain RL training approach and reveals several key aspects affecting cross-domain transferability. We further train two models {\ours}-7B and {\ours}-32B purely with RL on our curated data and observe largely improved performance over leading open RL reasoning model baselines, with gains of 7.3\% and 7.8\% respectively on an extensive 17-task, six-domain evaluation suite. We are releasing our dataset, code, and evaluation suite to the community, aiming to support further research and development of more general RL-enhanced reasoning models.