Skip to content
Papers.

Attention Is All You Need research paper by Google, 2017

Google · Jun 12, 2017 · Architectures · 193,621 citations · 142 upvotes

Read on arXiv

What it shows

Introduced the Transformer, the attention-only architecture behind nearly every large language model since.

Summarised by hand from the abstract.

More from Google

All 4
PaperCitations
RRSI: Regularized Recursive Self-Improvement of Agent HarnessesRRSI keeps self-improving agent harnesses from memorising their training tasks, so gains carry over to new benchmarks.Agents and evaluation · Sep 20260
Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsAsking a model to write out its intermediate steps (chain of thought) sharply improves its math and logic answers.Reasoning · Jan 202222.1k
BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingPre-training a Transformer to fill in masked words, then fine-tuning it, beat task-specific models across language understanding tests.Foundation models · Oct 2018121k

Other architectures papers

Topic
PaperCitations
Mamba: Linear-Time Sequence Modeling with Selective State SpacesA selective state space model that scales linearly with sequence length and matches Transformers on language.Carnegie Mellon University · Dec 20239,182
On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training StabilityQwen3.8-Flash-Next: a 125B mixture-of-experts model with 6B active that nearly matches its 397B predecessor at 1/9 the training compute.Alibaba (Qwen) · Aug 20266
About this paper
Authors
Ashish Vaswani, Noam Shazeer, Niki Parmar and 5 more
arXiv
1706.03762 · PDF
Venue
Neural Information Processing Systems
Citations
193,621, 20,774 influential · Semantic Scholar
Upvotes
142 · Hugging Face
Lab
Google · on Companies · on Acquisitions · on Paydays · on TechConf · on Releases

Changes

What changed
New paperAdded to the listSep 24, 2026

Sources: each lab's own papers and arXiv, with citation and upvote counts from Semantic Scholar and Hugging Face. One-line summaries are for orientation, not a substitute for the paper. Logos via logo.dev; trademarks belong to their owners.

New papers by email

Monday afternoons, only in weeks with new papers from the labs.

Double opt-in. Unsubscribe any time.