SpecForge: A Flexible and Efficient Open-Source Training Framework for Speculative Decoding
Nanyang Technological University · Meituan · Baseten · Together AI · Beijing University of Post and Telecommunications · LunaLab · Alibaba Group · EIGENAI · University of Wisconsin - Madison · Nanyang Technological University, Singapore · A*STAR
PDF 由论文原始站点提供，PaperCompass 不保存论文文件。
摘要
Speculative decoding mitigates the memory-bound nature of LLM decoding by using a lightweight draft model to propose multiple tokens for parallel verification. However, its adoption has been limited by the lack of high-quality draft models and scalable training infrastructure. We introduce SpecForge, an open-source and efficient framework for training speculative decoding models with full support for EAGLE-3. SpecForge incorporates target–draft decoupling, hybrid parallelism, optimized training kernels, and tight integration with production-grade inference engines, enabling up to 9.9x faster EAGLE-3 training for Qwen3-235B-A22B compared to the baseline. We further release SpecBundle, a suite of production-grade EAGLE-3 draft models trained with SpecForge for mainstream open-source LLMs, achieving up to 4.48x end-to-end inference speedup on SGLang and addressing the scarcity of high-quality drafts. Finally, we distill a systematic study of speculative decoding training into practical and actionable recipes to guide real-world adoption.