Papersby GlyphVibe Checker: Aligning Code Evaluation with Human Preference research paper by Google DeepMind, 2025
Google DeepMind · Agents and evaluation · Oct 8, 2025 · 2 citations · 34 upvotes · unverified
What it shows
Vibe Checker evaluates LLMs by combining functional correctness and instruction following to better align with human coding preferences.
By Ming Zhong, Xiang Zhou, Ting-Yun Chang and 9 more · arXiv 2510.07315 · PDF
UnverifiedHugging Face's summary; not yet checked by hand.
More from Google DeepMind
All 12DiffusionGemma Technical ReportDiffusionGemma is a fine-tuned mixture-of-experts language model that uses discrete diffusion to generate text blocks in parallel, achieving high speed while preserving capabilities like multimodal inputs and reasoning.
Google DeepMind14 citations40 upvotesRead →Gemma 4 Technical ReportOpen multimodal models from 2.3B to 31B parameters, with a thinking mode and image and audio input.Google DeepMind356 citations81 upvotesRead →LiteFrame: Efficient Vision Encoders Unlock Frame Scaling in Video LLMsLiteFrame, a lightweight video encoder with Compressed Token Distillation training method, reduces latency and increases frame processing capacity for long-form video understanding in Video LLMs while maintaining accuracy.Google DeepMind2 citations26 upvotesRead →Understanding the Challenges in Iterative Generative Optimization with LLMsGenerative optimization using large language models faces challenges due to implicit design decisions about artifact modification and learning evidence that significantly impact success across different applications.Google DeepMind11 citations27 upvotesRead →LoGeR: Long-Context Geometric Reconstruction with Hybrid MemoryLoGeR enables long-term 3D video reconstruction by combining bidirectional priors with a hybrid memory system that includes parametric Test-Time Training and non-parametric sliding window attention mechanisms.Google DeepMind46 citations63 upvotesRead →SIMA 2: A Generalist Embodied Agent for Virtual WorldsSIMA 2, built on a Gemini foundation model, interacts in 3D virtual worlds, reasons about goals, handles complex instructions, and autonomously learns new skills through open-ended self-improvement.Google DeepMind27 citations26 upvotesRead →Robot Learning from a Physical World ModelPhysWorld integrates video generation and physical world modeling to enable accurate robotic manipulation from visual demonstrations without real robot data.Google DeepMind23 citations32 upvotesRead →Video models are zero-shot learners and reasonersVeo 3, a generative video model, exhibits zero-shot capabilities across various visual tasks, suggesting a trajectory towards becoming a unified, generalist vision foundation model.Google DeepMind237 citations100 upvotesRead →
Other agents and evaluation papers
TopicGemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic CapabilitiesGemini 2.X model family, including Gemini 2.5 Pro and Flash, offers superior coding, reasoning, and multimodal understanding capabilities across a range of computational efficiencies.
Google4,594 citations67 upvotesRead →DeepSeek-V3.2: Pushing the Frontier of Open Large Language ModelsDeepSeek-V3.2 introduces DeepSeek Sparse Attention and a scalable reinforcement learning framework, achieving superior reasoning and performance compared to GPT-5 and Gemini-3.0-Pro in complex reasoning tasks.
DeepSeek887 citations275 upvotesRead →WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?WorkArena and BrowserGym evaluate large language model-based agents' ability to perform enterprise software tasks, revealing gaps in current agent capabilities and differences between open and closed-source LLMs.
ServiceNow443 citations2 upvotesRead →Aya Model: An Instruction Finetuned Open-Access Multilingual Language ModelAya, a multilingual generative language model supporting over 50% lower-resourced languages, excels in both generative and discriminative tasks across 99 languages, and provides extensive evaluations and open-source resources.
Cohere438 citations48 upvotesRead →Rubrics as Rewards: Reinforcement Learning Beyond Verifiable DomainsRubrics as Rewards (RaR) framework uses structured rubrics as interpretable reward signals for on-policy training, improving performance in real-world reinforcement learning tasks with subjective criteria.
Scale AI431 citations6 upvotesRead →SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?SWE-Bench Pro is a challenging benchmark for coding models, featuring complex, enterprise-level problems that require substantial code modifications, with performance evaluations showing significant limitations in current models.Scale AI367 citations21 upvotesRead →
About this paper
- Citations
- 2, 0 influential · Semantic Scholar
- Upvotes
- 34 · Hugging Face
- Lab
- Google DeepMind · on Companies · on Acquisitions
Changes
Citations rose from 1 to 2.
Citations fell from 2 to 1.
A first count of 0 influential citations.
A first count of 2 citations.
Found by the weekly Hugging Face scan; not yet checked by hand.