Notable data and AI papers

Papers627Labs45

627 papers · 45 labs

The Llama 3 Herd of Models research paper by Meta, 2024

Meta · Foundation models · Jul 31, 2024 · 20,834 citations · 119 upvotes

Read on arXiv

What it shows

Open models up to 405B parameters that match leading closed models on many tasks.

By Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey and 530 more · arXiv 2407.21783 · PDF

Summarised by hand from the abstract.

The model it introduced

On models.fru.dev
ModelLlama 3.1 8bmodels.fru.devBenchmarks →

More from Meta

All 40
Multimodal and roboticsUnverifiedOct 25, 2026InteractiveWorld: A Verifiable Human-Centric Environment for Building Better Interactive MLLM AssistantsInteractiveWorld provides a verifiable simulation environment and benchmark to evaluate whether multimodal language models can reliably guide humans through physical tasks.Meta46 upvotesRead →Multimodal and roboticsUnverifiedOct 2, 2026Native Action-Prior Learning from Videos for World Action ModelsNAVA-WAM learns robot action priors directly from observation-only videos, improving control performance without needing action-labeled data for pretraining.Meta88 upvotesRead →Alignment and safetyUnverifiedOct 1, 2026Sharpening Tax in Post-TrainingThis paper introduces a metric to quantify the loss in solution coverage after reinforcement learning and proposes a sampler to mitigate it.Meta1 citations111 upvotesRead →Multimodal and roboticsUnverifiedOct 1, 2026World Action Modeling with Progressive Visual PlanningProWAM improves robotic control by predicting sparse visual sub-goals to guide action generation, avoiding the need for dense video generation.Meta1 citations98 upvotesRead →ArchitecturesUnverifiedSep 29, 2026Context Language ModelsContext Language Models manage their own context by treating it as an editable file, improving performance and efficiency.Meta5 citations47 upvotesRead →Inference and efficiencyUnverifiedSep 26, 2026In-Flight KV Cache with Clean Anchors for Faster Autoregressive Video DiffusionFlashForward accelerates video diffusion by reusing in-flight KV caches and using sparse clean anchors to maintain structural consistency.Meta1 citations34 upvotesRead →Agents and evaluationUnverifiedSep 4, 2026WearableQA: A Benchmark for Health Reasoning over Real-World Wearable DataWearableQA is a benchmark of multiple-choice questions derived from real longitudinal wearable data that evaluates large language model reasoning across data and health dimensions.Meta43 upvotesRead →Multimodal and roboticsAug 6, 2026Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and RecipesControlled experiments on when text, image understanding and image generation help or compete when trained together.Meta7 citations63 upvotesRead →

Other foundation models papers

Topic
Foundation modelsOct 11, 2018BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingPre-training a Transformer to fill in masked words, then fine-tuning it, beat task-specific models across language understanding tests.Google125k citations33 upvotesRead →Foundation modelsMay 28, 2020Language Models are Few-Shot LearnersGPT-3, a 175B-parameter model, does new tasks from a few examples in the prompt, with no fine-tuning.OpenAI66.4k citations21 upvotesRead →Foundation modelsMar 15, 2023GPT-4 Technical ReportA multimodal model that passes a simulated bar exam with a score around the top 10% of test takers.OpenAI28.4k citations7 upvotesRead →Foundation modelsUnverifiedMay 14, 2025Qwen3 Technical ReportQwen3, a unified series of large language models, integrates thinking and non-thinking modes, reduces computational resources, and achieves state-of-the-art performance across various tasks and languages.Alibaba (Qwen)10.1k citations339 upvotesRead →Foundation modelsMar 8, 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextRecalls details across millions of tokens of text, hours of video and days of audio.Google DeepMind4,190 citations65 upvotesRead →Foundation modelsUnverifiedNov 26, 2025Qwen3-VL Technical ReportQwen3-VL, a vision-language model, excels in text and multimodal understanding through advanced architectures and larger contexts, achieving superior performance across benchmarks.Alibaba (Qwen)3,178 citations164 upvotesRead →
About this paper
Citations
20,834, 3,790 influential · Semantic Scholar
Upvotes
119 · Hugging Face
Lab
Meta · on Companies · on Acquisitions · on Quarterly · on Paydays · on TechConf

Changes

CitationsNov 9, 2026Influential citations rose from 3,731 to 3,790.
CitationsNov 9, 2026Citations rose from 20,541 to 20,834.
CitationsNov 2, 2026Influential citations rose from 3,589 to 3,731.
CitationsNov 2, 2026Citations rose from 19,641 to 20,541.
CitationsOct 12, 2026Influential citations rose from 3,538 to 3,589.
CitationsOct 12, 2026Citations rose from 19,314 to 19,641.
CitationsOct 5, 2026Influential citations rose from 3,471 to 3,538.
CitationsOct 5, 2026Citations rose from 18,933 to 19,314.
CitationsSep 28, 2026Influential citations rose from 3,457 to 3,471.
CitationsSep 28, 2026Citations rose from 18,856 to 18,933.
CitationsSep 25, 2026Influential citations rose from 3,453 to 3,457.
CitationsSep 25, 2026Citations rose from 18,841 to 18,856.
New paperSep 24, 2026Added to the list.