Papersby Glyph

Notable data and AI papers

Papers627 Labs45

627 papers · 45 labs

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding research paper by Google, 2018

Google · Foundation models · Oct 11, 2018 · 125,374 citations · 33 upvotes

Pre-training a Transformer to fill in masked words, then fine-tuning it, beat task-specific models across language understanding tests.

By Jacob Devlin, Ming-Wei Chang, Kenton Lee and 1 more · arXiv 1810.04805 · PDF

Summarised by hand from the abstract.

More from Google

Inference and efficiencyUnverifiedOct 27, 2026EmbeddingGemma 2: Unified and Efficient Multimodal RepresentationsEmbeddingGemma 2 is an open multimodal embedding model that unifies text, images, audio, and video representations with efficient resource-flexible design and state-of-the-art benchmark performance.

Google34 upvotesRead →Multimodal and roboticsUnverifiedOct 26, 2026StreamVibe: Proactive Video Agents for Evolving User IntentStreamVibe introduces a benchmark, streaming vision-language model, and hierarchical memory harness to enable proactive, intent-aware interaction with implicit and evolving user goals.

Google78 upvotesRead →Training and scalingUnverifiedOct 23, 2026On-Policy Self-Training: Data-Free Self-Improvement in Large Language ModelsFoundation models can self-improve across text and vision tasks by generating synthetic questions and reasoning traces guided only by high-level topics, without external supervision.

Google31 upvotesRead →Training and scalingUnverifiedOct 14, 2026DeLoop: Repeated Actions as Self-Supervision for Long-Horizon AgentsThis self-supervised training method teaches AI agents to avoid repetitive, unproductive actions during long tasks by using their own history as feedback.

Google21 upvotesRead →Agents and evaluationUnverifiedOct 13, 2026CLASHarness: Benchmarking Multi-Agent Conflict in Computer Use via a Controllable and Verifiable Generation PipelineThis paper introduces a benchmark and generation pipeline to evaluate how agents resolve conflicts and verify information in multi-agent computer-use scenarios.

Google36 upvotesRead →Agents and evaluationUnverifiedOct 9, 2026Harness^2: Recursive Agent Harnessing for an Open WorldThis paper introduces a recursive framework that allows agents to automatically improve their own execution harnesses for better performance on novel open-world tasks.

Google2 citations123 upvotesRead →Multimodal and roboticsUnverifiedOct 7, 2026VIS-Ground: Video Interactive Storytelling with Contextual GroundingThis paper introduces a framework that transforms heterogeneous user inputs into structured constraints to improve grounding in interactive video generation.

Google103 upvotesRead →Applied AIUnverifiedOct 2, 2026SEER: Self-Evolving Event Reasoning and Retrieval for Time Series ForecastingSEER improves time series forecasting by using a closed-loop system to retrieve and reason about external events that influence data trends.

Google35 upvotesRead →

Foundation modelsMay 28, 2020Language Models are Few-Shot LearnersGPT-3, a 175B-parameter model, does new tasks from a few examples in the prompt, with no fine-tuning.

OpenAI66.4k citations21 upvotesRead →Foundation modelsMar 15, 2023GPT-4 Technical ReportA multimodal model that passes a simulated bar exam with a score around the top 10% of test takers.

OpenAI28.4k citations7 upvotesRead →Foundation modelsFeb 27, 2023LLaMA: Open and Efficient Foundation Language ModelsOpen models from 7B to 65B trained only on public data; the 13B model beats GPT-3 on most benchmarks.

Meta22.6k citations25 upvotesRead →Foundation modelsJul 31, 2024The Llama 3 Herd of ModelsOpen models up to 405B parameters that match leading closed models on many tasks.

Meta20.8k citations119 upvotesRead →Foundation modelsUnverifiedMay 14, 2025Qwen3 Technical ReportQwen3, a unified series of large language models, integrates thinking and non-thinking modes, reduces computational resources, and achieves state-of-the-art performance across various tasks and languages.

Alibaba (Qwen)10.1k citations339 upvotesRead →Foundation modelsMar 8, 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextRecalls details across millions of tokens of text, hours of video and days of audio.

Google DeepMind4,190 citations65 upvotesRead →

About this paper

Venue: North American Chapter of the Association for Computational Linguistics
Citations: 125,374, 23,518 influential · Semantic Scholar
Upvotes: 33 · Hugging Face
Lab: Google · on Companies · on Acquisitions · on Paydays · on TechConf · on Releases

Changes

CitationsNov 9, 2026Influential citations rose from 23,515 to 23,518.

CitationsNov 9, 2026Citations rose from 124,847 to 125,374.

CitationsNov 2, 2026Influential citations rose from 23,114 to 23,515.

CitationsNov 2, 2026Citations rose from 122,538 to 124,847.

CitationsOct 12, 2026Influential citations rose from 23,014 to 23,114.

CitationsOct 12, 2026Citations rose from 121,524 to 122,538.

CitationsSep 28, 2026Influential citations rose from 22,983 to 23,014.

CitationsSep 28, 2026Citations rose from 121,375 to 121,524.

CitationsSep 25, 2026Influential citations rose from 22,971 to 22,983.

CitationsSep 25, 2026Citations rose from 121,314 to 121,375.

New paperSep 24, 2026Added to the list.