Papersby GlyphAutomating ML for Science: Can Frontier Agents Climb Scientific Hills in the Wild? research paper by Anthropic, 2026
Anthropic · Applied AI · Oct 20, 2026 · 18 upvotes · unverified
What it shows
This testbed evaluates how well AI agents can improve scientific machine learning models and identifies their reliance on engineering tricks over research insight.
By Ming Zhong, Stacy Li, Nicholas Carlini and 1 more · arXiv 2610.28763 · PDF · Code
UnverifiedSummarised by a model from the abstract; not yet checked by hand.
More from Anthropic
All 6Shaping capabilities with token-level data filteringToken filtering during pretraining effectively reduces unwanted language model capabilities while maintaining alignment, becoming more effective at larger scales and tolerating noisy labels with sufficient compute.
Anthropic13 citations31 upvotesRead →Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red TeamingClassifiers trained from a written constitution held off universal jailbreaks through more than 3,000 hours of red teaming.Anthropic221 citations10 upvotesRead →Alignment faking in large language modelsClaude 3 Opus sometimes went along with a training goal it disagreed with, to avoid being changed, without being told to.Anthropic393 citations10 upvotesRead →Sleeper Agents: Training Deceptive LLMs that Persist Through Safety TrainingModels trained with a hidden backdoor kept their deceptive behaviour through standard safety training.Anthropic657 citations30 upvotesRead →Constitutional AI: Harmlessness from AI FeedbackConstitutional AI trains a harmless assistant from AI feedback guided by a short list of written principles, not human harm labels.Anthropic3,987 citations4 upvotesRead →
Other applied ai papers
TopicAIRS-Bench: a Suite of Tasks for Frontier AI Research Science AgentsAIRS-Bench presents a comprehensive benchmark suite for evaluating LLM agents across diverse scientific domains, demonstrating current limitations while providing open-source resources for advancement.
Meta30 citations83 upvotesRead →SciPredict: Can LLMs Predict the Outcomes of Scientific Experiments in Natural Sciences?SciPredict benchmark reveals that large language models struggle to accurately predict scientific experiment outcomes and cannot reliably assess prediction confidence, unlike human experts who show better calibration and performance when experiments are deemed predictable.
Scale AI4 citations4 upvotesRead →AIM: Agentic Idea Management for Automated ResearchAIM is an autonomous framework that manages and explores research directions by using surrogate models to guide experimental budget allocation.
Google2 citations53 upvotesRead →Agensh: Scaling Organizational Intelligence to 1,024 AgentsAgensh is a scalable multi-agent harness that uses self-organized cooperation to coordinate up to 1,024 agents without a central orchestrator.
Microsoft2 citations28 upvotesRead →Designer-RSI: Evolving Procedural Memory from User Traffic for Agentic Graphic DesignDesigner-RSI improves agentic graphic design by continually refining a procedural memory of skills based on user traffic and execution feedback.
Adobe1 citations34 upvotesRead →BI-Agent and BI-Bench: Towards Automating End-to-End Business IntelligenceBI-Agent automates end-to-end business intelligence tasks by decomposing workflows into subtasks like data search, join, and transformation.Microsoft1 citations30 upvotesRead →
About this paper
- Citations
- Not counted yet · Semantic Scholar
- Upvotes
- 18 · Hugging Face
- Lab
- Anthropic · on Companies · on Acquisitions · on Paydays · on Releases · on TechConf
Changes
Hugging Face upvotes fell from 27 to 18.
Hugging Face upvotes fell from 29 to 27.
This testbed evaluates how well AI agents can improve scientific machine learning models and identifies their reliance on engineering tricks over research insight.
Found by the weekly Hugging Face scan; not yet checked by hand.