Papersby GlyphSciPredict: Can LLMs Predict the Outcomes of Scientific Experiments in Natural Sciences? research paper by Scale AI, 2026
Scale AI · Applied AI · Apr 12, 2026 · 4 citations · 4 upvotes
What it shows
SciPredict benchmark reveals that large language models struggle to accurately predict scientific experiment outcomes and cannot reliably assess prediction confidence, unlike human experts who show better calibration and performance when experiments are deemed predictable.
By Udari Madhushani Sehwag, Elaine Lau, Haniyeh Ehsani Oskouie and 14 more · arXiv 2604.10718 · PDF
Hugging Face's summary; not yet checked by hand.
More from Scale AI
All 25Root-Cause Attribution Is a Search Problem: Continual Search for Long-Horizon Agent FailuresContinual Search improves automated root-cause attribution in long agent execution traces by iteratively prompting diagnosis until unresolved evidence is found.
Scale AI5 upvotesRead →SteerDuplex: Steerable Duplex Speech Dialogue ModelsA full-duplex speech model that follows spoken instructions on tone, persona, pace and voice, with a benchmark of 390 prompts to test it.Scale AI3 citations6 upvotesRead →Studying Without a Syllabus: Task-Agnostic Environment PreprocessingAn agent can explore unfamiliar environments without task-specific guidance to build reusable artifacts that reduce later inference costs, though larger study budgets do not always improve results.Scale AI1 citations4 upvotesRead →HarnessOpt-Bench: Evaluating LLMs at Harness OptimizationHarnessOpt-Bench measures how well frontier LLMs iteratively improve agent harnesses under constrained evaluation budgets, revealing substantial variation across models and tasks.Scale AI12 citations35 upvotesRead →Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent FailuresAn interaction-centric taxonomy localizes agent failures to specific component interactions to guide targeted repairs across diverse architectures.Scale AI11 citations10 upvotesRead →SWE-INTERACT: Reimagining SWE Benchmarks as User-Driven Long-Horizon Coding SessionsSWE-Interact presents a testbed that evaluates coding agents in realistic multi-turn, user-driven software engineering scenarios, revealing significant gaps between single-turn performance and interactive task completion.Scale AI23 citations7 upvotesRead →Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVRPOW3R is a policy-aware framework for reinforcement learning with rubric-based rewards that adapts criterion weights during training to improve policy optimization while preserving human-defined criteria importance.Scale AI13 citations5 upvotesRead →Reward Hacking in Rubric-Based Reinforcement LearningResearch examines reward hacking in rubric-based reinforcement learning, identifying verifier failure and rubric-design limitations as key sources of divergence between training and evaluation metrics.Scale AI33 citations4 upvotesRead →
Other applied ai papers
TopicAIRS-Bench: a Suite of Tasks for Frontier AI Research Science AgentsAIRS-Bench presents a comprehensive benchmark suite for evaluating LLM agents across diverse scientific domains, demonstrating current limitations while providing open-source resources for advancement.
Meta30 citations83 upvotesRead →AIM: Agentic Idea Management for Automated ResearchAIM is an autonomous framework that manages and explores research directions by using surrogate models to guide experimental budget allocation.
Google2 citations53 upvotesRead →Agensh: Scaling Organizational Intelligence to 1,024 AgentsAgensh is a scalable multi-agent harness that uses self-organized cooperation to coordinate up to 1,024 agents without a central orchestrator.
Microsoft2 citations28 upvotesRead →Designer-RSI: Evolving Procedural Memory from User Traffic for Agentic Graphic DesignDesigner-RSI improves agentic graphic design by continually refining a procedural memory of skills based on user traffic and execution feedback.
Adobe1 citations34 upvotesRead →BI-Agent and BI-Bench: Towards Automating End-to-End Business IntelligenceBI-Agent automates end-to-end business intelligence tasks by decomposing workflows into subtasks like data search, join, and transformation.Microsoft1 citations30 upvotesRead →StudentSim: Training LLM-based Student SimulatorsStudentSim trains per-student simulators that answer like a given learner and change their answers under a tutor's guidance.Microsoft1 citations371 upvotesRead →
About this paper
- Venue
- arXiv.org
- Citations
- 4, 0 influential · Semantic Scholar
- Upvotes
- 4 · Hugging Face
- Lab
- Scale AI · on Companies · on Acquisitions · on awards · on breaches · on Attested · on Rounds · on Downtime · on labs · on Leaders · on marketplaces · on Allied · on Paydays · on quotes · on salaries · on Nines · on socials · on startups · on terms · on Titlewave
Changes
Citations rose from 3 to 4.
Citations rose from 2 to 3.
Citations rose from 1 to 2.
A first count of 0 influential citations.
A first count of 1 citations.
Added to the list.