RAFT (Reward ranked Fine-Tuning Algorithm)
What it is
RAFT is an official implementation of an iterative best-of-n fine-tuning and rejection sampling approach. It works by generating multiple responses per prompt, ranking them using a reward/preference model, and then fine-tuning the model on the highest-ranked responses.
The problem it solves
Reduces the noise and low-quality data often found in standard SFT by using a reward-ranking mechanism to select only the best responses.
What you could use it for
- Fine-tuning Large Language Models (LLMs) using iterative reward ranking.
- Generating high-quality data for SFT (Supervised Fine-Tuning) from multiple model outputs.
- Implementing rejection sampling for model alignment.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.