Skip to content
Papers.

Language Models are Few-Shot Learners research paper by OpenAI, 2020

OpenAI · May 28, 2020 · Foundation models · 63,622 citations · 21 upvotes

Read on arXiv

What it shows

GPT-3, a 175B-parameter model, does new tasks from a few examples in the prompt, with no fine-tuning.

Summarised by hand from the abstract.

More from OpenAI

All 5
PaperCitations
Why Language Models HallucinateModels hallucinate because training and benchmarks reward confident guessing over saying they do not know.Alignment and safety · Sep 2025329
GPT-4 Technical ReportA multimodal model that passes a simulated bar exam with a score around the top 10% of test takers.Foundation models · Mar 202327.3k
Training language models to follow instructions with human feedbackInstructGPT: fine-tuning on human feedback made a 1.3B model preferred over the 175B GPT-3.Alignment and safety · Mar 202224.2k
Scaling Laws for Neural Language ModelsLanguage model loss falls as a smooth power law as model size, data and compute grow.Training and scaling · Jan 20209,061

Other foundation models papers

Topic
PaperCitations
BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingPre-training a Transformer to fill in masked words, then fine-tuning it, beat task-specific models across language understanding tests.Google · Oct 2018121k
LLaMA: Open and Efficient Foundation Language ModelsOpen models from 7B to 65B trained only on public data; the 13B model beats GPT-3 on most benchmarks.Meta · Feb 202321.7k
The Llama 3 Herd of ModelsOpen models up to 405B parameters that match leading closed models on many tasks.Meta · Jul 202418.8k
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextRecalls details across millions of tokens of text, hours of video and days of audio.Google DeepMind · Mar 20244,002
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your PhoneSmall models trained on filtered and synthetic data that run on a phone and rival much larger ones.Microsoft · Apr 20242,497
Nemotron-4 340B Technical ReportOpen models sized for one 8-GPU server, aligned with over 98% synthetic data.NVIDIA · Jun 2024138
About this paper
Authors
Tom B. Brown, Benjamin Mann, Nick Ryder and 28 more
arXiv
2005.14165 · PDF
Venue
Neural Information Processing Systems
Citations
63,622, 5,497 influential · Semantic Scholar
Upvotes
21 · Hugging Face
Lab
OpenAI · on Companies · on Acquisitions · on Paydays · on Releases · on TechConf

Changes

What changed
New paperAdded to the listSep 24, 2026

Sources: each lab's own papers and arXiv, with citation and upvote counts from Semantic Scholar and Hugging Face. One-line summaries are for orientation, not a substitute for the paper. Logos via logo.dev; trademarks belong to their owners.

New papers by email

Monday afternoons, only in weeks with new papers from the labs.

Double opt-in. Unsubscribe any time.