Skip to content
Papers.

Training language models to follow instructions with human feedback research paper by OpenAI, 2022

OpenAI · Mar 4, 2022 · Alignment and safety · 24,241 citations · 26 upvotes

Read on arXiv

What it shows

InstructGPT: fine-tuning on human feedback made a 1.3B model preferred over the 175B GPT-3.

Summarised by hand from the abstract.

More from OpenAI

All 5
PaperCitations
Why Language Models HallucinateModels hallucinate because training and benchmarks reward confident guessing over saying they do not know.Alignment and safety · Sep 2025329
GPT-4 Technical ReportA multimodal model that passes a simulated bar exam with a score around the top 10% of test takers.Foundation models · Mar 202327.3k
Language Models are Few-Shot LearnersGPT-3, a 175B-parameter model, does new tasks from a few examples in the prompt, with no fine-tuning.Foundation models · May 202063.6k
Scaling Laws for Neural Language ModelsLanguage model loss falls as a smooth power law as model size, data and compute grow.Training and scaling · Jan 20209,061

Other alignment and safety papers

Topic
PaperCitations
Direct Preference Optimization: Your Language Model is Secretly a Reward ModelDPO aligns a model to human preferences with a simple classification loss, no reward model or RL loop.Stanford University · May 202310.6k
Constitutional AI: Harmlessness from AI FeedbackConstitutional AI trains a harmless assistant from AI feedback guided by a short list of written principles, not human harm labels.Anthropic · Dec 20223,718
Sleeper Agents: Training Deceptive LLMs that Persist Through Safety TrainingModels trained with a hidden backdoor kept their deceptive behaviour through standard safety training.Anthropic · Jan 2024576
Alignment faking in large language modelsClaude 3 Opus sometimes went along with a training goal it disagreed with, to avoid being changed, without being told to.Anthropic · Dec 2024342
Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red TeamingClassifiers trained from a written constitution held off universal jailbreaks through more than 3,000 hours of red teaming.Anthropic · Jan 2025199
About this paper
Authors
Long Ouyang, Jeff Wu, Xu Jiang and 17 more
arXiv
2203.02155 · PDF
Venue
Neural Information Processing Systems
Citations
24,241, 2,459 influential · Semantic Scholar
Upvotes
26 · Hugging Face
Code
github.com/openai/following-instructions-human-feedback
Lab
OpenAI · on Companies · on Acquisitions · on Paydays · on Releases · on TechConf

Changes

What changed
New paperAdded to the listSep 24, 2026

Sources: each lab's own papers and arXiv, with citation and upvote counts from Semantic Scholar and Hugging Face. One-line summaries are for orientation, not a substitute for the paper. Logos via logo.dev; trademarks belong to their owners.

New papers by email

Monday afternoons, only in weeks with new papers from the labs.

Double opt-in. Unsubscribe any time.