Papersby GlyphWebscale-RL: Automated Data Pipeline for Scaling RL Data to Pretraining Levels research paper by Salesforce, 2025
Salesforce · Training and scaling · Oct 7, 2025 · 9 citations · 33 upvotes
What it shows
A scalable data engine converts large-scale pre-training documents into diverse question-answer pairs for reinforcement learning, significantly improving model performance and efficiency.
By Zhepeng Cen, Haolin Chen, Shiyu Wang and 7 more · arXiv 2510.06499 · PDF · Code
Hugging Face's summary; not yet checked by hand.
More from Salesforce
All 34Self-Retrospection Distillation: Turning Post-hoc Experiences into Prior ForesightSelf-Retrospection Distillation improves agent performance by using hindsight from completed trajectories to supervise foresight predictions during training.
Salesforce1 citations197 upvotesRead →Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM AgentsKeeps an agent's raw past runs and writes a task-specific memory only when a new task arrives, rather than deciding up front what to keep.Salesforce2 citations56 upvotesRead →Flattening Every Memory Peak in Long-Context Mixture-of-Experts TrainingBounds the four memory peaks that crash long-context mixture-of-experts training, from expert dispatch to optimizer state, with fixed-size GPU schedules.Salesforce19 upvotesRead →RISE: Recursive Improvement via Self-Extrapolating Policy DistillationRISE improves language model post-training by recursively generating dense token-level supervision from the model's own reinforcement learning trajectory via self-extrapolation, avoiding external teachers.Salesforce5 citations17 upvotesRead →EVOHARNESSBENCH: Can Your Agents Keep Pace with an Evolving Harness?The study introduces a benchmark to evaluate LLM agents under evolving tool, skill, and agent harnesses, revealing persistent gaps in retention, adaptation, and harness-induced forgetting.Salesforce7 citations32 upvotesRead →Random Attention: Rethinking KV Cache Eviction for Efficient ReasoningRandom eviction of reasoning tokens matches selective KV cache compression because reasoning traces are self-protecting through redundancy, making scoring unnecessary once prompts are preserved.Salesforce5 citations187 upvotesRead →DarwinX: Evolving Agent Harnesses Through Natural SelectionDarwinX evolves agent harnesses via population selection with frozen models, improving verified performance across benchmarks without benchmark-specific patches.Salesforce16 citations116 upvotesRead →StateAct: Program State, before Pixels, for Long-Horizon Computer-Use AgentsStateAct improves computer-use agents by grounding actions, verification, and memory in direct program state rather than screenshots, boosting success rates while reducing cost.Salesforce6 citations62 upvotesRead →
Other training and scaling papers
TopicLoRA: Low-Rank Adaptation of Large Language ModelsLoRA fine-tunes a large model by training small low-rank matrices, cutting trainable parameters by 10,000 times.
Microsoft25.6k citations66 upvotesRead →Scaling Laws for Neural Language ModelsLanguage model loss falls as a smooth power law as model size, data and compute grow.
OpenAI9,592 citations12 upvotesRead →Training Compute-Optimal Large Language ModelsChinchilla: for a fixed compute budget, train a smaller model on more data; parameters and tokens should grow together.
Google DeepMind4,035 citations13 upvotesRead →Megatron-LM: Training Multi-Billion Parameter Language Models Using Model ParallelismMegatron-LM splits each Transformer layer across GPUs to train models with billions of parameters.
NVIDIA3,387 citations5 upvotesRead →DeepSeek LLM: Scaling Open-Source Language Models with LongtermismDeepSeek LLM, an open-source language model project, develops a large dataset and employs SFT and DPO to achieve performance surpassing LLaMA-2 70B and GPT-3.5 in various benchmarks and open-ended evaluations.
DeepSeek872 citations56 upvotesRead →LoRA Learns Less and Forgets LessLoRA learns less than full fine-tuning on code and math, but forgets less of what the model already knew.
Databricks486 citations91 upvotesRead →
About this paper
- Venue
- International Conference on Learning Representations
- Citations
- 9, 5 influential · Semantic Scholar
- Upvotes
- 33 · Hugging Face
- Lab
- Salesforce · on Companies · on Acquisitions · on Quarterly · on Paydays · on TechConf
Changes
Influential citations rose from 4 to 5.
Influential citations rose from 3 to 4.
Citations rose from 8 to 9.
Citations rose from 7 to 8.
A first count of 3 influential citations.
A first count of 7 citations.
Added to the list.