Skip to content
Papers.

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness research paper by Stanford University, 2022

Stanford University · May 27, 2022 · Inference and efficiency · 5,331 citations · 16 upvotes

Read on arXiv

What it shows

FlashAttention computes exact attention with far fewer GPU memory reads and writes, making long sequences faster.

Summarised by hand from the abstract.

More from Stanford University

All 2
PaperCitations
Direct Preference Optimization: Your Language Model is Secretly a Reward ModelDPO aligns a model to human preferences with a simple classification loss, no reward model or RL loop.Alignment and safety · May 202310.6k

Other inference and efficiency papers

Topic
PaperCitations
Efficient Memory Management for Large Language Model Serving with PagedAttentionvLLM manages the KV cache like pages of virtual memory, serving models with 2 to 4 times the throughput.UC Berkeley · Sep 20238,469
SwiftKV: Fast Prefill-Optimized Inference with Knowledge-Preserving Model TransformationSwiftKV lets prompt tokens skip later layers and merges KV caches, cutting prefill cost for long-prompt workloads.Snowflake · Oct 202418
DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache CompressionA 552B mixture-of-experts model with a 1M-token context, built to shrink the KV cache for agent workloads.DeepSeek · Sep 20267
About this paper
Authors
Tri Dao, Daniel Y. Fu, Stefano Ermon and 2 more
arXiv
2205.14135 · PDF
Venue
Neural Information Processing Systems
Citations
5,331, 451 influential · Semantic Scholar
Upvotes
16 · Hugging Face
Lab
Stanford University

Changes

What changed
New paperAdded to the listSep 24, 2026

Sources: each lab's own papers and arXiv, with citation and upvote counts from Semantic Scholar and Hugging Face. One-line summaries are for orientation, not a substitute for the paper. Logos via logo.dev; trademarks belong to their owners.

New papers by email

Monday afternoons, only in weeks with new papers from the labs.

Double opt-in. Unsubscribe any time.