Papersby Glyph

Notable data and AI papers

Papers627 Labs45

627 papers · 45 labs

Attention Is All You Need research paper by Google, 2017

Google · Architectures · Jun 12, 2017 · 200,859 citations · 142 upvotes

Introduced the Transformer, the attention-only architecture behind nearly every large language model since.

By Ashish Vaswani, Noam Shazeer, Niki Parmar and 5 more · arXiv 1706.03762 · PDF

Summarised by hand from the abstract.

More from Google

Inference and efficiencyUnverifiedOct 27, 2026EmbeddingGemma 2: Unified and Efficient Multimodal RepresentationsEmbeddingGemma 2 is an open multimodal embedding model that unifies text, images, audio, and video representations with efficient resource-flexible design and state-of-the-art benchmark performance.

Google34 upvotesRead →Multimodal and roboticsUnverifiedOct 26, 2026StreamVibe: Proactive Video Agents for Evolving User IntentStreamVibe introduces a benchmark, streaming vision-language model, and hierarchical memory harness to enable proactive, intent-aware interaction with implicit and evolving user goals.

Google78 upvotesRead →Training and scalingUnverifiedOct 23, 2026On-Policy Self-Training: Data-Free Self-Improvement in Large Language ModelsFoundation models can self-improve across text and vision tasks by generating synthetic questions and reasoning traces guided only by high-level topics, without external supervision.

Google31 upvotesRead →Training and scalingUnverifiedOct 14, 2026DeLoop: Repeated Actions as Self-Supervision for Long-Horizon AgentsThis self-supervised training method teaches AI agents to avoid repetitive, unproductive actions during long tasks by using their own history as feedback.

Google21 upvotesRead →Agents and evaluationUnverifiedOct 13, 2026CLASHarness: Benchmarking Multi-Agent Conflict in Computer Use via a Controllable and Verifiable Generation PipelineThis paper introduces a benchmark and generation pipeline to evaluate how agents resolve conflicts and verify information in multi-agent computer-use scenarios.

Google36 upvotesRead →Agents and evaluationUnverifiedOct 9, 2026Harness^2: Recursive Agent Harnessing for an Open WorldThis paper introduces a recursive framework that allows agents to automatically improve their own execution harnesses for better performance on novel open-world tasks.

Google2 citations123 upvotesRead →Multimodal and roboticsUnverifiedOct 7, 2026VIS-Ground: Video Interactive Storytelling with Contextual GroundingThis paper introduces a framework that transforms heterogeneous user inputs into structured constraints to improve grounding in interactive video generation.

Google103 upvotesRead →Applied AIUnverifiedOct 2, 2026SEER: Self-Evolving Event Reasoning and Retrieval for Time Series ForecastingSEER improves time series forecasting by using a closed-loop system to retrieve and reason about external events that influence data trends.

Google35 upvotesRead →

ArchitecturesDec 1, 2023Mamba: Linear-Time Sequence Modeling with Selective State SpacesA selective state space model that scales linearly with sequence length and matches Transformers on language.

Carnegie Mellon University10.2k citations152 upvotesRead →ArchitecturesSep 2, 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and GenerationCodeT5, a unified encoder-decoder Transformer model, improves code understanding and generation by leveraging semantic information from identifiers and user comments.

Salesforce2,633 citations3 upvotesRead →ArchitecturesUnverifiedMar 14, 2025VGGT: Visual Geometry Grounded TransformerVGGT, a feed-forward neural network, efficiently infers multiple 3D attributes from single or multiple views, outperforming alternatives and enhancing downstream tasks without post-processing.

Meta2,132 citations40 upvotesRead →ArchitecturesUnverifiedJan 11, 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language ModelsThe DeepSeekMoE architecture improves expert specialization in Mixture-of-Experts models by segmenting experts and isolating shared ones, achieving better performance and computational efficiency compared to GShard and other models.

DeepSeek1,268 citations65 upvotesRead →ArchitecturesMay 13, 2023CodeT5+: Open Code Large Language Models for Code Understanding and GenerationCodeT5+, a family of flexible encoder-decoder LLMs initialized with frozen LLMs and trained with a mixture of pretraining objectives, achieves state-of-the-art performance across various code-related tasks.

Salesforce836 citations7 upvotesRead →ArchitecturesUnverifiedOct 9, 2019HuggingFace's Transformers: State-of-the-art Natural Language ProcessingTransformers library provides state-of-the-art Transformer architectures and pretrained models for natural language processing tasks with a unified API and emphasis on extensibility and robust deployment.

Hugging Face251 citations34 upvotesRead →

About this paper

Venue: Neural Information Processing Systems
Citations: 200,859, 21,427 influential · Semantic Scholar
Upvotes: 142 · Hugging Face
Lab: Google · on Companies · on Acquisitions · on Paydays · on TechConf · on Releases

Changes

CitationsNov 2, 2026Influential citations rose from 20,931 to 21,427.

CitationsNov 2, 2026Citations rose from 196,494 to 200,859.

CitationsOct 12, 2026Influential citations rose from 20,863 to 20,931.

CitationsOct 12, 2026Citations rose from 195,201 to 196,494.

CitationsOct 5, 2026Influential citations rose from 20,799 to 20,863.

CitationsOct 5, 2026Citations rose from 194,086 to 195,201.

CitationsSep 28, 2026Influential citations rose from 20,778 to 20,799.

CitationsSep 28, 2026Citations rose from 193,768 to 194,086.

CitationsSep 25, 2026Influential citations rose from 20,774 to 20,778.

CitationsSep 25, 2026Citations rose from 193,621 to 193,768.

New paperSep 24, 2026Added to the list.