Skip to content
Papers.

An RL View of OPD: Least Square Policy Distillation for Sample-Efficient LLM Reasoning research paper by NVIDIA, 2026

NVIDIA · Sep 28, 2026 · Reasoning · 25 upvotes · unverified 7 days ago

Read on arXiv

What it shows

We study on-policy distillation (OPD) through the lens of reinforcement learning, establishing a connection between the reverse-KL objective in OPD and KL-regularized policy optimization.

By Shangzhe Li, Yuxiao Yang, Tianrun Yu and 4 more · arXiv 2609.35505 · PDF · Code

UnverifiedHugging Face's summary; not yet checked by hand.

More from NVIDIA

All 79
PaperCitations
Mid-Harness: Scaling Actions Between Model and Harness for Terminal AgentsTerminal agents act through stochastic model generations, yet the ability to generate a useful action does not ensure its reliable execution.Agents and evaluation · Sep 2026 · Unverified5 days ago-
LongLive-Plug: Once-for-All Distillation for Video GenerationVideo diffusion models are increasingly developed into specialized models for diverse downstream tasks, and this development often includes a distillation stage, for example to accelerate sampling or to improve...Multimodal and robotics · Sep 2026 · Unverified6 days ago-
PixelUMM: Encoder-Free Unified Image and Video Understanding and GenerationUnified Multimodal Models (UMMs) often rely on separate visual representations for understanding and generation, increasing visual context length and complicating integration with established vision-language...Multimodal and robotics · Sep 2026 · Unverified6 days ago-
SoL-Pi: Recursively Scaling Auto-Research Loops for Efficient Agent HarnessAs coding agents move from supervised code completion to unattended, around-the-clock exploration, their work expands from isolated predictions into long trajectories of reasoning, tool use, and feedback.Agents and evaluation · Sep 2026 · Unverified2 weeks ago4
An Open Recipe for IMO Gold: Training Nemotron for Olympiad MathematicsA natural-language proof-generation pipeline using post-trained Nemotron 3 Ultra checkpoints achieves gold-medal performance on IMO 2026 through iterative verification and refinement without external tools.Training and scaling · Sep 2026 · Unverified3 weeks ago0
Online Draft Co-Training for Speculative Decoding in Large-Scale, Long-Context RL Post-TrainingA system for large-scale online draft co-training accelerates speculative decoding in RL post-training by extending context-parallel attention and adding cross-stage feature transport.Training and scaling · Sep 2026 · Unverified4 weeks ago2
Sol-Attn: Accelerating Video Generation Inference via On-the-Fly Attention SparsificationSol-Attn improves training-free sparse attention for diffusion transformers by combining dynamic block routing, sparse computation, and approximation correction in a single online pass to accelerate video generation without sacrificing quality.Inference and efficiency · Jul 2026 · Unverified2 months ago6
SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video GenerationSANA-Video 2.0 is a hybrid video diffusion transformer that combines linear and softmax attention to generate high-resolution video efficiently on a single GPU.Inference and efficiency · Jul 2026 · Unverified2 months ago2

Other reasoning papers

Topic
PaperCitations
Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsAsking a model to write out its intermediate steps (chain of thought) sharply improves its math and logic answers.Google · Jan 20224 years ago22.5k
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language ModelsDeepSeekMath 7B improves mathematical reasoning through enhanced data pre-training and Group Relative Policy Optimization, achieving high scores on MATH benchmark without external tools.DeepSeek · Feb 2024 · Unverified2 years ago9,498
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement LearningReinforcement learning taught a model long step-by-step reasoning on par with OpenAI o1.DeepSeek · Jan 20251 year ago5,805
DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code IntelligenceDeepSeek-Coder-V2, a Mixture-of-Experts language model, excels in code-specific tasks by enhancing coding and mathematical reasoning capabilities while expanding language support and context length.DeepSeek · Jun 2024 · Unverified2 years ago507
MolmoAct: Action Reasoning Models that can Reason in SpaceAction Reasoning Models (ARMs) integrate perception, planning, and control to enable adaptable and explainable robotic behavior, achieving superior performance across various tasks and settings.Ai2 · Aug 2025 · Unverified1 year ago199
WebShaper: Agentically Data Synthesizing via Information-Seeking FormalizationWebShaper, a formalization-driven framework, synthesizes information-seeking datasets using set theory and Knowledge Projections to enhance reasoning structure and achieve top performance in open-sourced benchmarks.Alibaba (Qwen) · Jul 2025 · Unverified1 year ago115
About this paper
Authors
Shangzhe Li, Yuxiao Yang, Tianrun Yu and 4 more
arXiv
2609.35505 · PDF
Citations
Not counted yet · Semantic Scholar
Upvotes
25 · Hugging Face
Code
github.com/UNCSciML/LSPD
Lab
NVIDIA · on Companies · on Acquisitions · on Quarterly · on Paydays · on TechConf

Changes

What changed
New paperFound by the weekly scan, unverifiedOct 5, 2026today

New papers by email

Monday afternoons, only in weeks with new papers from the labs.

Double opt-in. Unsubscribe any time.