Training Language Models to Follow Instructions with Human Feedback
OpenAI · · Research
OpenAI fine-tuned GPT-3 with human feedback (RLHF) to follow instructions. The 1.3B-parameter InstructGPT model was preferred by human evaluators over the 175B GPT-3.
Why it matters
The alignment tax became the alignment dividend. Fine-tuning with human feedback made language models actually useful — not just capable at next-token prediction, but capable at doing what humans want. Every chat product after this uses this technique.
This is LLM World's historical reading, not a claim by OpenAI. The facts above and the sources below are what it rests on.
Model line behind it: GPT.
Sources
Record status: verified · confidence 95/100
arXiv paper 2203.02155, submitted 2022-03-04. The paper is the primary source. Also published at NeurIPS 2022.