NEFTune: Noisy Embeddings Improve Instruction Finetuning
University of Maryland, College Park · University of Maryland - College Park · University of Maryland · Department of Computer Science, University of Maryland, College Park · Lawrence Livermore National Laboratory · Lawrence Livermore National Lab · Carnegie Mellon University · New York University (NYU) · ELLIS Institute & MPI Intelligent Systems, Tübingen AI Center
PDF 由论文原始站点提供，PaperCompass 不保存论文文件。
摘要
We show that language model finetuning can be improved, sometimes dramatically, with a simple augmentation. NEFTune adds noise to the embedding vectors during training.Standard finetuning of LLaMA-2-7B using Alpaca achieves $29.79$\% on AlpacaEval, which rises to $64.69$\% using noisy embeddings. NEFTune also improves over strong baselines on modern instruction datasets.Models trained with Evol-Instruct see a $10$\% improvement, with ShareGPT an $8$\% improvement, and with OpenPlatypus an $8$\% improvement. Even powerful models further refined with RLHF such as LLaMA-2-Chat benefit from additional training with NEFTune. Particularly, we see these improvements on the conversational abilities of the instruction model and not on traditional tasks like those on the OpenLLM Leaderboard, where performance is the same.