Papersby GlyphTraining and scaling
Scaling laws, fine-tuning, distributed training and optimisation. 48 papers from 18 labs.
On-Policy Self-Training: Data-Free Self-Improvement in Large Language ModelsFoundation models can self-improve across text and vision tasks by generating synthetic questions and reasoning traces guided only by high-level topics, without external supervision.
Google31 upvotesRead →Self-Evolving Coding Agents via Harness-Policy OptimizationThis framework improves coding agents by recursively alternating between optimizing the agent's policy and the environment harness it uses to interact with code.
Microsoft48 upvotesRead →DeLoop: Repeated Actions as Self-Supervision for Long-Horizon AgentsThis self-supervised training method teaches AI agents to avoid repetitive, unproductive actions during long tasks by using their own history as feedback.Google21 upvotesRead →APT: Feedback-Driven Agentic Post-Training for Text-to-Image GenerationThis paper introduces an automated post-training framework that uses a feedback loop of evaluation and diagnosis to improve text-to-image models.Microsoft23 upvotesRead →When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often BetterThis paper shows that distilling from offline student rollouts is often more efficient and accurate than standard on-policy distillation for many model pairs.
NVIDIA28 upvotesRead →QwenGyre: An Elastic Reinforcement Learning Framework for Training xLong-Horizon AgentsQwenGyre is an online reinforcement learning framework that elastically manages GPU resources to train agents on extremely long-horizon tasks.
Alibaba (Qwen)2 citations43 upvotesRead →Selecting Diverse SFT Traces Improves Post-RL GeneralizationSelecting diverse reasoning paths during supervised fine-tuning improves model performance and generalization after reinforcement learning.Google38 upvotesRead →What masking geometry works best for EEG foundation models?Spatio-temporal masking strategies for EEG foundation models were systematically compared across MAE and JEPA frameworks, revealing robust performance across configurations and a novel JEPA failure mode, with optimal masking achieving high downstream accuracy at reduced compute.
Hugging Face3 upvotesRead →One to More, More to One: Category-Aware Iterative Expert Training for Software Engineering AgentsThis framework improves software engineering agents by training category-specific experts and consolidating them into a single student model.Alibaba (Qwen)50 upvotesRead →Don't Mask the Environment: Observation Supervision Changes How Agents Explore Under RLActObs improves agent reinforcement learning by supervising the model to predict environment observations during supervised fine-tuning.
Amazon44 upvotesRead →When EOS Tokens Disagree: Understanding Length Inflation in On-Policy DistillationThis paper identifies termination-token mismatch as a cause of length inflation in policy distillation and provides a mitigation strategy.Microsoft6 citations109 upvotesRead →An Open Recipe for IMO Gold: Training Nemotron for Olympiad MathematicsA natural-language proof-generation pipeline using post-trained Nemotron 3 Ultra checkpoints achieves gold-medal performance on IMO 2026 through iterative verification and refinement without external tools.NVIDIA43 upvotesRead →Online Draft Co-Training for Speculative Decoding in Large-Scale, Long-Context RL Post-TrainingA system for large-scale online draft co-training accelerates speculative decoding in RL post-training by extending context-parallel attention and adding cross-stage feature transport.NVIDIA4 citations35 upvotesRead →RISE: Recursive Improvement via Self-Extrapolating Policy DistillationRISE improves language model post-training by recursively generating dense token-level supervision from the model's own reinforcement learning trajectory via self-extrapolation, avoiding external teachers.
Salesforce5 citations17 upvotesRead →Language Models Need Sleep: Learning to Self-Modify and Consolidate MemoriesDeep learning models with sleep and dreaming paradigms enable continual learning through memory consolidation and self-improvement phases.Google9 citations28 upvotesRead →Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVRPOW3R is a policy-aware framework for reinforcement learning with rubric-based rewards that adapts criterion weights during training to improve policy optimization while preserving human-defined criteria importance.
Scale AI13 citations5 upvotesRead →The Illusion of Certainty: Decoupling Capability and Calibration in On-Policy DistillationOn-policy distillation suffers from miscalibration due to information mismatch between training and deployment contexts, which is addressed through a calibration-aware framework that improves both performance and confidence reliability.Salesforce10 citations14 upvotesRead →Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy DistillationNemotron-Cascade 2 is a 30B parameter Mixture-of-Experts model with 3B activated parameters that achieves exceptional reasoning and agentic capabilities, matching frontier open models despite its compact size and demonstrating high intelligence density.NVIDIA63 citations70 upvotesRead →Online Experiential Learning for Language ModelsOnline Experiential Learning enables continuous improvement of language models through deployment experience by extracting and consolidating experiential knowledge via on-policy distillation.Microsoft28 citations61 upvotesRead →Tiny Aya: Bridging Scale and Multilingual DepthTiny Aya demonstrates high-quality multilingual capabilities with 3.35 billion parameters through region-aware posttraining and balanced language performance.
Cohere18 citations10 upvotesRead →On Data Engineering for Scaling LLM Terminal CapabilitiesResearchers developed a synthetic task generation pipeline and analyzed data strategies to improve terminal agent performance, creating a large-scale dataset and models that outperform larger counterparts on benchmark tests.NVIDIA44 citations104 upvotesRead →Test-Time Training with KV Binding Is Secretly Linear AttentionTest-time training is reinterpreted as learned linear attention rather than memorization, offering architectural simplifications and improved efficiency.NVIDIA16 citations32 upvotesRead →Unified Latents (UL): How to train your latentsUnified Latents framework learns joint latent representations using diffusion prior regularization and diffusion model decoding, achieving competitive FID scores with reduced training compute.Google19 citations63 upvotesRead →MSign: An Optimizer Preventing Training Instability in Large Language Models via Stable Rank RestorationTraining instability in large language models is linked to weight matrix stable rank decline and Jacobian alignment, which MSign addresses through matrix sign operations to prevent gradient explosions.Microsoft2 citations34 upvotesRead →Residual Context Diffusion Language ModelsResidual Context Diffusion (RCD) enhances diffusion large language models by recycling discarded token information through contextual residuals, improving accuracy with minimal computational overhead.
UC Berkeley13 citations36 upvotesRead →Shaping capabilities with token-level data filteringToken filtering during pretraining effectively reduces unwanted language model capabilities while maintaining alignment, becoming more effective at larger scales and tolerating noisy labels with sufficient compute.
Anthropic13 citations31 upvotesRead →Jet-RL: Enabling On-Policy FP8 Reinforcement Learning with Unified Training and Rollout Precision FlowQuantized reinforcement learning training using FP8 precision faces stability issues due to numerical mismatches between training and inference phases, but a unified FP8 framework achieves significant speedups with stable convergence.NVIDIA18 citations26 upvotesRead →mHC: Manifold-Constrained Hyper-ConnectionsManifold-Constrained Hyper-Connections (mHC) stabilize and scale residual connection architectures by restoring identity mapping properties through manifold projection and infrastructure optimization.
DeepSeek108 citations334 upvotesRead →Masking Teacher and Reinforcing Student for Distilling Vision-Language ModelsMasters is a reinforcement learning distillation framework that progressively restores teacher capacity to help small students learn richer representations.NVIDIA7 citations30 upvotesRead →SignRoundV2: Closing the Performance Gap in Extremely Low-Bit Post-Training Quantization for LLMsSignRoundV2, a post-training quantization framework, achieves competitive accuracy for Large Language Models at extremely low-bit quantization through layer-wise bit allocation and pre-tuning scale search.
Intel1 citations17 upvotesRead →Souper-Model: How Simple Arithmetic Unlocks State-of-the-Art LLM PerformanceSoCE is a model souping approach that uses benchmark composition to identify and combine expert models for improved performance.
Meta4 citations140 upvotesRead →BitNet DistillationBitNet Distillation fine-tunes large language models to 1.58-bit precision using SubLN, multi-head attention distillation, and continual pre-training, achieving comparable performance with significant memory and inference speed improvements.Microsoft2 citations62 upvotesRead →The Art of Scaling Reinforcement Learning Compute for LLMsA systematic study defines a framework for analyzing and predicting reinforcement learning scaling in large language models, identifying key design choices that affect compute efficiency and proposing a best-practice recipe.Meta112 citations34 upvotesRead →Webscale-RL: Automated Data Pipeline for Scaling RL Data to Pretraining LevelsA scalable data engine converts large-scale pre-training documents into diverse question-answer pairs for reinforcement learning, significantly improving model performance and efficiency.Salesforce9 citations33 upvotesRead →RLP: Reinforcement as a Pretraining ObjectiveRLP, an information-driven reinforcement pretraining objective, enhances reasoning models by integrating exploration into pretraining, leading to significant performance improvements across various benchmarks.NVIDIA27 citations47 upvotesRead →Chasing the Tail: Effective Rubric-based Reward Modeling for Large Language Model Post-TrainingRubric-based rewards mitigate reward over-optimization in reinforcement fine-tuning by leveraging off-policy examples while maintaining reward reliability.Scale AI62 citations20 upvotesRead →Verification Limits Code LLM TrainingVerification design and strategies impact code generation performance, showing that richer and more diverse test suites improve capabilities while calibrated verification thresholds enhance data usability.Cohere2 citationsRead →Jointly Reinforcing Diversity and Quality in Language Model GenerationsDARLING, a diversity-aware reinforcement learning framework, enhances both the quality and diversity of large language model outputs across various tasks.Meta92 citations25 upvotesRead →Demystifying Domain-adaptive Post-training for Financial LLMsFINDAP, a systematic investigation, improves LLMs for finance through continual pretraining, instruction tuning, and preference alignment, achieving state-of-the-art performance using a novel preference data distillation method.Salesforce30 citations11 upvotesRead →jina-embeddings-v3: Multilingual Embeddings With Task LoRAjina-embeddings-v3, a large-scale text embedding model, achieves state-of-the-art performance in multilingual and long-context retrieval tasks using Low-Rank Adaptation and Matryoshka Representation Learning.
Jina AI221 citations35 upvotesRead →To Code, or Not To Code? Exploring Impact of Code in Pre-trainingThe inclusion of code data during pre-training significantly improves LLMs' performance across various natural language and world knowledge tasks, beyond code generation.Cohere61 citations45 upvotesRead →LoRA Learns Less and Forgets LessLoRA learns less than full fine-tuning on code and math, but forgets less of what the model already knew.
Databricks486 citations91 upvotesRead →DeepSeek LLM: Scaling Open-Source Language Models with LongtermismDeepSeek LLM, an open-source language model project, develops a large dataset and employs SFT and DPO to achieve performance surpassing LLaMA-2 70B and GPT-3.5 in various benchmarks and open-ended evaluations.DeepSeek872 citations56 upvotesRead →When Less is More: Investigating Data Pruning for Pretraining LLMs at ScalePerplexity outperforms more complex methods for data quality estimation in pruning large language model training datasets, resulting in performance improvement with reduced data.Cohere144 citations17 upvotesRead →Training Compute-Optimal Large Language ModelsChinchilla: for a fixed compute budget, train a smaller model on more data; parameters and tokens should grow together.
Google DeepMind4,035 citations13 upvotesRead →LoRA: Low-Rank Adaptation of Large Language ModelsLoRA fine-tunes a large model by training small low-rank matrices, cutting trainable parameters by 10,000 times.Microsoft25.6k citations66 upvotesRead →Scaling Laws for Neural Language ModelsLanguage model loss falls as a smooth power law as model size, data and compute grow.
OpenAI9,592 citations12 upvotesRead →Megatron-LM: Training Multi-Billion Parameter Language Models Using Model ParallelismMegatron-LM splits each Transformer layer across GPUs to train models with billions of parameters.NVIDIA3,387 citations5 upvotesRead →
Amazon44 upvotesRead →When EOS Tokens Disagree: Understanding Length Inflation in On-Policy DistillationThis paper identifies termination-token mismatch as a cause of length inflation in policy distillation and provides a mitigation strategy.