Skip to content
Papers.

The Llama 3 Herd of Models research paper by Meta, 2024

Meta · Jul 31, 2024 · Foundation models · 18,841 citations · 119 upvotes

Read on arXiv

What it shows

Open models up to 405B parameters that match leading closed models on many tasks.

Summarised by hand from the abstract.

More from Meta

All 4
PaperCitations
Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and RecipesControlled experiments on when text, image understanding and image generation help or compete when trained together.Multimodal and robotics · Aug 20262
Segment AnythingA promptable model and a dataset of over a billion masks that cut out any object in any image.Multimodal and robotics · Apr 202315.7k
LLaMA: Open and Efficient Foundation Language ModelsOpen models from 7B to 65B trained only on public data; the 13B model beats GPT-3 on most benchmarks.Foundation models · Feb 202321.7k

Other foundation models papers

Topic
PaperCitations
BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingPre-training a Transformer to fill in masked words, then fine-tuning it, beat task-specific models across language understanding tests.Google · Oct 2018121k
Language Models are Few-Shot LearnersGPT-3, a 175B-parameter model, does new tasks from a few examples in the prompt, with no fine-tuning.OpenAI · May 202063.6k
GPT-4 Technical ReportA multimodal model that passes a simulated bar exam with a score around the top 10% of test takers.OpenAI · Mar 202327.3k
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextRecalls details across millions of tokens of text, hours of video and days of audio.Google DeepMind · Mar 20244,002
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your PhoneSmall models trained on filtered and synthetic data that run on a phone and rival much larger ones.Microsoft · Apr 20242,497
Nemotron-4 340B Technical ReportOpen models sized for one 8-GPU server, aligned with over 98% synthetic data.NVIDIA · Jun 2024138
About this paper
Authors
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey and 530 more
arXiv
2407.21783 · PDF
Citations
18,841, 3,453 influential · Semantic Scholar
Upvotes
119 · Hugging Face
Lab
Meta · on Companies · on Acquisitions · on Quarterly · on Paydays · on TechConf

Changes

What changed
New paperAdded to the listSep 24, 2026

Sources: each lab's own papers and arXiv, with citation and upvote counts from Semantic Scholar and Hugging Face. One-line summaries are for orientation, not a substitute for the paper. Logos via logo.dev; trademarks belong to their owners.

New papers by email

Monday afternoons, only in weeks with new papers from the labs.

Double opt-in. Unsubscribe any time.