Papersby GlyphBlack-Box On-Policy Distillation of Large Language Models research paper by Microsoft, 2025
Microsoft · Alignment and safety · Nov 13, 2025 · 63 citations · 54 upvotes · unverified
What it shows
Generative Adversarial Distillation (GAD) enhances black-box distillation by framing the student model as a generator and using a discriminator to provide adaptive feedback, surpassing traditional sequence-level knowledge distillation.
By Tianzhu Ye, Li Dong, Zewen Chi and 3 more · arXiv 2511.10643 · PDF
UnverifiedHugging Face's summary; not yet checked by hand.
More from Microsoft
All 57WebHarbor: Docking Real Websites for Evolving GUI Agent EnvironmentsWebHarbor provides realistic, containerized website environments for training and testing computer-using web agents across complex online tasks.
Microsoft28 upvotesRead →Agent World Model in the Wild: Computer-Use Agent Environment SynthesisThis pipeline creates controllable, open-source environments to safely train and test AI agents on complex, long-horizon computer-use tasks.Microsoft69 upvotesRead →Self-Evolving Coding Agents via Harness-Policy OptimizationThis framework improves coding agents by recursively alternating between optimizing the agent's policy and the environment harness it uses to interact with code.Microsoft48 upvotesRead →AesCode: Aesthetic Code Generation with Decoupled Cross-Modal RewardsThis paper introduces a method to improve code-based visual generation by using image-based aesthetic guidance and decoupled rewards to ensure quality and correctness.Microsoft137 upvotesRead →APT: Feedback-Driven Agentic Post-Training for Text-to-Image GenerationThis paper introduces an automated post-training framework that uses a feedback loop of evaluation and diagnosis to improve text-to-image models.Microsoft23 upvotesRead →LensErase: A Foundation Model for Erase AnythingThis paper introduces a foundation model designed for versatile image editing and object removal using mask-based inputs and efficient positional encoding.Microsoft52 upvotesRead →Simplifying and Improving On-Policy Distillation for Few-Step DiffusionThis paper introduces a simplified regression-based method for distilling diffusion models that improves generation speed without requiring complex training techniques.Microsoft27 upvotesRead →MiniCorp: The Last Mile of the AI Agent FirmMiniCorp is an office simulator that allows researchers to study how agents run companies and generate enterprise data for training.Microsoft39 upvotesRead →
Other alignment and safety papers
TopicTraining language models to follow instructions with human feedbackInstructGPT: fine-tuning on human feedback made a 1.3B model preferred over the 175B GPT-3.
OpenAI25.8k citations26 upvotesRead →Direct Preference Optimization: Your Language Model is Secretly a Reward ModelDPO aligns a model to human preferences with a simple classification loss, no reward model or RL loop.
Stanford University11.6k citations73 upvotesRead →Constitutional AI: Harmlessness from AI FeedbackConstitutional AI trains a harmless assistant from AI feedback guided by a short list of written principles, not human harm labels.
Anthropic3,987 citations4 upvotesRead →Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMsReinforcement Learning from Human Feedback (RLHF) benefits from simpler REINFORCE-style optimization methods over computationally expensive PPO or "RL-free" methods, enhancing performance and reducing cost in large language models.
Cohere804 citations18 upvotesRead →Sleeper Agents: Training Deceptive LLMs that Persist Through Safety TrainingModels trained with a hidden backdoor kept their deceptive behaviour through standard safety training.Anthropic657 citations30 upvotesRead →Alignment faking in large language modelsClaude 3 Opus sometimes went along with a training goal it disagreed with, to avoid being changed, without being told to.Anthropic393 citations10 upvotesRead →
About this paper
- Venue
- arXiv.org
- Citations
- 63, 7 influential · Semantic Scholar
- Upvotes
- 54 · Hugging Face
- Lab
- Microsoft · on Companies · on Acquisitions · on Quarterly · on Paydays · on Releases · on TechConf
Changes
Citations rose from 62 to 63.
Influential citations rose from 4 to 7.
Citations rose from 49 to 62.
Influential citations rose from 3 to 4.
Citations rose from 47 to 49.
A first count of 3 influential citations.
A first count of 47 citations.
Found by the weekly Hugging Face scan; not yet checked by hand.