Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning
University of Science and Technology of China · University of Oxford · Imperial College London · WanFlow AI and Wuhan University · The Chinese University of Hong Kong and Shanghai Artificial Intelligence Laboratory · National University of Singapore · University of Oxford and Stanford University · Shanghai AI Laboratory
PDF 由论文原始站点提供，PaperCompass 不保存论文文件。DOI 10.18653/v1/2026.acl-long.1444 ↗
摘要
While scaling laws for large language models (LLMs) during pre-training have been extensively studied, their behavior under reinforcement learning (RL) post-training remains largely unexplored. This paper investigates the scaling behavior of Large Language Model (LLM) reinforcement learning post-training, focusing on mathematical reasoning. Through experiments across the Qwen2.5 series (0.5B to 72B), we characterize how model scale, data, and compute interact. Our analysis yields four key findings: 1. Larger models consistently demonstrate superior compute and data efficiency. 2. The relationship between model performance and training resources follows a **predictive power-law** across both base and instruction-tuned models. 3. RL learning efficiency exhibits a latent **saturation trend** with increasing model scale. 4. In data-constrained regimes, performance is primarily driven by the **total volume of training data** rather than sample uniqueness. These results offer practical guidelines for scaling reasoning capabilities through reinforcement learning post-training.