Papersby GlyphPlanning In Natural Language Improves LLM Search For Code Generation research paper by Scale AI, 2024
Scale AI · Retrieval and data · Sep 5, 2024 · 101 citations
What it shows
PLANSEARCH, a novel search algorithm, improves performance in coding benchmarks by generating a diverse set of natural language plans, outperforming existing methods through increased diversity in solutions.
By Evan Wang, Federico Cassano, Catherine Wu and 7 more · arXiv 2409.03733 · PDF
Hugging Face's summary; not yet checked by hand.
More from Scale AI
All 25Root-Cause Attribution Is a Search Problem: Continual Search for Long-Horizon Agent FailuresContinual Search improves automated root-cause attribution in long agent execution traces by iteratively prompting diagnosis until unresolved evidence is found.
Scale AI5 upvotesRead →SteerDuplex: Steerable Duplex Speech Dialogue ModelsA full-duplex speech model that follows spoken instructions on tone, persona, pace and voice, with a benchmark of 390 prompts to test it.Scale AI3 citations6 upvotesRead →Studying Without a Syllabus: Task-Agnostic Environment PreprocessingAn agent can explore unfamiliar environments without task-specific guidance to build reusable artifacts that reduce later inference costs, though larger study budgets do not always improve results.Scale AI1 citations4 upvotesRead →HarnessOpt-Bench: Evaluating LLMs at Harness OptimizationHarnessOpt-Bench measures how well frontier LLMs iteratively improve agent harnesses under constrained evaluation budgets, revealing substantial variation across models and tasks.Scale AI12 citations35 upvotesRead →Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent FailuresAn interaction-centric taxonomy localizes agent failures to specific component interactions to guide targeted repairs across diverse architectures.Scale AI11 citations10 upvotesRead →SWE-INTERACT: Reimagining SWE Benchmarks as User-Driven Long-Horizon Coding SessionsSWE-Interact presents a testbed that evaluates coding agents in realistic multi-turn, user-driven software engineering scenarios, revealing significant gaps between single-turn performance and interactive task completion.Scale AI23 citations7 upvotesRead →Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVRPOW3R is a policy-aware framework for reinforcement learning with rubric-based rewards that adapts criterion weights during training to improve policy optimization while preserving human-defined criteria importance.Scale AI13 citations5 upvotesRead →Reward Hacking in Rubric-Based Reinforcement LearningResearch examines reward hacking in rubric-based reinforcement learning, identifying verifier failure and rubric-design limitations as key sources of divergence between training and evaluation metrics.Scale AI33 citations4 upvotesRead →
Other retrieval and data papers
TopicBLOOM: A 176B-Parameter Open-Access Multilingual Language ModelBLOOM, a 176B-parameter open-access language model, achieves competitive performance on various benchmarks and is released publicly with a Responsible AI License.
Hugging Face3,165 citations41 upvotesRead →From Local to Global: A Graph RAG Approach to Query-Focused SummarizationGraphRAG builds a knowledge graph of a document set so a model can answer questions about the whole collection.
Microsoft2,561 citations7 upvotesRead →DINOv3DINOv3, a self-supervised learning model, achieves superior performance across various vision tasks by scaling datasets and models, addressing dense feature degradation, and enhancing flexibility with post-hoc strategies.
Meta1,932 citations320 upvotesRead →Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and RankingThe Qwen3-VL-Embedding and Qwen3-VL-Reranker models form an end-to-end multimodal search pipeline, leveraging multi-stage training and cross-attention mechanisms to achieve high-precision retrieval across diverse modalities.
Alibaba (Qwen)315 citations59 upvotesRead →DeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic DataGenerating extensive Lean 4 proof data from mathematical competition problems improved DeepSeekMath 7B's theorem-proving capabilities over GPT-4 and other methods.
DeepSeek284 citations47 upvotesRead →Aya Dataset: An Open-Access Collection for Multilingual Instruction TuningThe initiative builds a human-curated instruction-following dataset spanning 65 languages and creates the largest multilingual collection of instruction-following instances through templating and translating existing datasets across 114 languages, contributing datasets and platforms for participatory research.
Cohere253 citations57 upvotesRead →
About this paper
- Venue
- arXiv.org
- Citations
- 101, 6 influential · Semantic Scholar
- Upvotes
- 0 · Hugging Face
- Lab
- Scale AI · on Companies · on Acquisitions · on awards · on breaches · on Attested · on Rounds · on Downtime · on labs · on Leaders · on marketplaces · on Allied · on Paydays · on quotes · on salaries · on Nines · on socials · on startups · on terms · on Titlewave
Changes
Citations rose from 98 to 101.
Citations rose from 95 to 98.
Citations rose from 93 to 95.
A first count of 6 influential citations.
A first count of 93 citations.
Added to the list.