Papersby GlyphAgents' Last Exam research paper by UC Berkeley, 2026
UC Berkeley · Agents and evaluation · Jun 3, 2026 · 42 citations · 393 upvotes
What it shows
Over 1,000 real, checkable professional tasks written with 250+ industry experts; the hardest tier is far from solved.
By Yiyou Sun, Xinyang Han, Weichen Zhang and 305 more · arXiv 2606.05405 · PDF · Code
Summarised by hand from the abstract.
More from UC Berkeley
All 14Correction Training for Diffusion ModelsThis method adds a correction step to diffusion models to fix errors that accumulate during the image generation process.
UC Berkeley42 upvotesRead →OmniTaskonomy: When Does Visual Generation Improve Visual Understanding?This paper maps how visual generation tasks improve visual understanding capabilities, revealing specific task-dependent transfer benefits.UC Berkeley57 upvotesRead →FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive ExecutionFreeToken is an edge-native Mixture-of-Experts serving system that dynamically maps computation and model state onto heterogeneous local hardware to run large open-weight models on personal machines.UC Berkeley8 citations86 upvotesRead →Playful Agentic Robot LearningEmbodied robots learn reusable skills through self-directed play and exploration, then apply these skills to improve performance on downstream tasks without additional training.UC Berkeley37 citations49 upvotesRead →Flash-KMeans: Fast and Memory-Efficient Exact K-MeansFlash-kmeans enables efficient online k-means clustering on GPUs through novel kernel-level optimizations that eliminate I/O bottlenecks and reduce atomic write contention.UC Berkeley16 citations64 upvotesRead →dLLM: Simple Diffusion Language ModelingA unified open-source framework is presented that standardizes core components of diffusion language modeling for reproduction, customization, and accessible development of both large and small models.UC Berkeley18 citations136 upvotesRead →SLA2: Sparse-Linear Attention with Learnable Routing and QATSLA2 improves sparse-linear attention in diffusion models by introducing a learnable router, direct attention formulation, and quantization-aware fine-tuning for enhanced efficiency and quality.UC Berkeley1 citations59 upvotesRead →Quant VideoGen: Auto-Regressive Long Video Generation via 2-Bit KV-Cache QuantizationQuant VideoGen addresses KV cache memory limitations in autoregressive video diffusion models through semantic-aware smoothing and progressive residual quantization, achieving significant memory reduction with minimal latency impact.UC Berkeley29 citations35 upvotesRead →
Other agents and evaluation papers
TopicGemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic CapabilitiesGemini 2.X model family, including Gemini 2.5 Pro and Flash, offers superior coding, reasoning, and multimodal understanding capabilities across a range of computational efficiencies.
Google4,594 citations67 upvotesRead →DeepSeek-V3.2: Pushing the Frontier of Open Large Language ModelsDeepSeek-V3.2 introduces DeepSeek Sparse Attention and a scalable reinforcement learning framework, achieving superior reasoning and performance compared to GPT-5 and Gemini-3.0-Pro in complex reasoning tasks.
DeepSeek887 citations275 upvotesRead →WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?WorkArena and BrowserGym evaluate large language model-based agents' ability to perform enterprise software tasks, revealing gaps in current agent capabilities and differences between open and closed-source LLMs.
ServiceNow443 citations2 upvotesRead →Aya Model: An Instruction Finetuned Open-Access Multilingual Language ModelAya, a multilingual generative language model supporting over 50% lower-resourced languages, excels in both generative and discriminative tasks across 99 languages, and provides extensive evaluations and open-source resources.
Cohere438 citations48 upvotesRead →Rubrics as Rewards: Reinforcement Learning Beyond Verifiable DomainsRubrics as Rewards (RaR) framework uses structured rubrics as interpretable reward signals for on-policy training, improving performance in real-world reinforcement learning tasks with subjective criteria.
Scale AI431 citations6 upvotesRead →SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?SWE-Bench Pro is a challenging benchmark for coding models, featuring complex, enterprise-level problems that require substantial code modifications, with performance evaluations showing significant limitations in current models.Scale AI367 citations21 upvotesRead →
About this paper
- Venue
- arXiv.org
- Citations
- 42, 4 influential · Semantic Scholar
- Upvotes
- 393 · Hugging Face
- Lab
- UC Berkeley
Changes
Influential citations rose from 3 to 4.
Citations rose from 37 to 42.
Influential citations rose from 1 to 3.
Citations rose from 25 to 37.
Citations rose from 21 to 25.
Citations rose from 15 to 21.
Added to the list.