Notable data and AI papers

Papers627Labs45

627 papers · 45 labs

ToolPRMBench: Evaluating and Advancing Process Reward Models for Tool-using Agents research paper by Intuit, 2026

Intuit · Agents and evaluation · Jan 18, 2026 · 18 citations · 19 upvotes · unverified

Read on arXiv

What it shows

ToolPRMBench is introduced as a large-scale benchmark for evaluating process reward models in tool-using agents, featuring step-level test cases and multi-LLM verification to ensure data quality.

By Dawei Li, Yuguang Yao, Zhen Tan and 2 more · arXiv 2601.12294 · PDF · Code

UnverifiedHugging Face's summary; not yet checked by hand.

More from Intuit

All 2
Retrieval and dataUnverifiedApr 7, 2026CUE-R: Beyond the Final Answer in Retrieval-Augmented GenerationResearchers developed a framework to measure the operational utility of individual retrieved items in retrieval-augmented generation systems by perturbing evidence and analyzing changes in correctness, grounding faithfulness, and confidence error.Intuit3 citations3 upvotesRead →

Other agents and evaluation papers

Topic
Agents and evaluationUnverifiedJul 7, 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic CapabilitiesGemini 2.X model family, including Gemini 2.5 Pro and Flash, offers superior coding, reasoning, and multimodal understanding capabilities across a range of computational efficiencies.Google4,594 citations67 upvotesRead →Agents and evaluationUnverifiedDec 2, 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language ModelsDeepSeek-V3.2 introduces DeepSeek Sparse Attention and a scalable reinforcement learning framework, achieving superior reasoning and performance compared to GPT-5 and Gemini-3.0-Pro in complex reasoning tasks.DeepSeek887 citations275 upvotesRead →Agents and evaluationMar 12, 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?WorkArena and BrowserGym evaluate large language model-based agents' ability to perform enterprise software tasks, revealing gaps in current agent capabilities and differences between open and closed-source LLMs.ServiceNow443 citations2 upvotesRead →Agents and evaluationFeb 12, 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language ModelAya, a multilingual generative language model supporting over 50% lower-resourced languages, excels in both generative and discriminative tasks across 99 languages, and provides extensive evaluations and open-source resources.Cohere438 citations48 upvotesRead →Agents and evaluationJul 23, 2025Rubrics as Rewards: Reinforcement Learning Beyond Verifiable DomainsRubrics as Rewards (RaR) framework uses structured rubrics as interpretable reward signals for on-policy training, improving performance in real-world reinforcement learning tasks with subjective criteria.Scale AI431 citations6 upvotesRead →Agents and evaluationSep 21, 2025SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?SWE-Bench Pro is a challenging benchmark for coding models, featuring complex, enterprise-level problems that require substantial code modifications, with performance evaluations showing significant limitations in current models.Scale AI367 citations21 upvotesRead →
About this paper
Venue
Annual Meeting of the Association for Computational Linguistics
Citations
18, 1 influential · Semantic Scholar
Upvotes
19 · Hugging Face
Lab
Intuit · on Companies · on Quarterly · on Paydays

Changes

CitationsNov 9, 2026Influential citations rose from 0 to 1.
CitationsNov 9, 2026Citations rose from 15 to 18.
CitationsNov 2, 2026Citations rose from 13 to 15.
CitationsOct 12, 2026A first count of 0 influential citations.
CitationsOct 12, 2026A first count of 13 citations.
New paperOct 12, 2026Found by the weekly Hugging Face scan; not yet checked by hand.