GALA: Geometric Data Selection with Strategic Prospecting for Large Language Model Self-training

摘要

Self-training has emerged as a promising direction for autonomously improving large language models (LLMs). Existing approaches typically adopt a \textit{generate-and-filter} paradigm based on rejection sampling, which could suffer from inefficiency and low-quality reasoning paths. Towards this end, this paper proposes a novel framework named \underline{G}eometric D\underline{a}ta Se\underline{l}ection with Str\underline{a}tegic Prospecting (GALA) for LLM self-training. The core of our GALA is to identify diverse and informative samples from redundant data and exploit them more strategically. In particular, our proposed GALA first conducts clustering on latent sentence embeddings and then selects an anchor sample from each cluster based on the geometric distance to reduce data redundancy. To further exploit these samples, we conduct strategic brainstorming and reflection for high-quality reasoning trajectory prospecting. In addition, we introduce a lightweight dynamic validation module to validate the reliability of mini-batches to ensure the overall quality of the data. Extensive experiments on various benchmarks validate the effectiveness of the proposed GALA against several competing baselines.