Papersby GlyphMSign: An Optimizer Preventing Training Instability in Large Language Models via Stable Rank Restoration research paper by Microsoft, 2026
Microsoft · Training and scaling · Feb 2, 2026 · 2 citations · 34 upvotes · unverified
What it shows
Training instability in large language models is linked to weight matrix stable rank decline and Jacobian alignment, which MSign addresses through matrix sign operations to prevent gradient explosions.
By Lianhai Ren, Yucheng Ding, Xiao Liu and 3 more · arXiv 2602.01734 · PDF
UnverifiedHugging Face's summary; not yet checked by hand.
More from Microsoft
All 57WebHarbor: Docking Real Websites for Evolving GUI Agent EnvironmentsWebHarbor provides realistic, containerized website environments for training and testing computer-using web agents across complex online tasks.
Microsoft28 upvotesRead →Agent World Model in the Wild: Computer-Use Agent Environment SynthesisThis pipeline creates controllable, open-source environments to safely train and test AI agents on complex, long-horizon computer-use tasks.Microsoft69 upvotesRead →Self-Evolving Coding Agents via Harness-Policy OptimizationThis framework improves coding agents by recursively alternating between optimizing the agent's policy and the environment harness it uses to interact with code.Microsoft48 upvotesRead →AesCode: Aesthetic Code Generation with Decoupled Cross-Modal RewardsThis paper introduces a method to improve code-based visual generation by using image-based aesthetic guidance and decoupled rewards to ensure quality and correctness.Microsoft137 upvotesRead →APT: Feedback-Driven Agentic Post-Training for Text-to-Image GenerationThis paper introduces an automated post-training framework that uses a feedback loop of evaluation and diagnosis to improve text-to-image models.Microsoft23 upvotesRead →LensErase: A Foundation Model for Erase AnythingThis paper introduces a foundation model designed for versatile image editing and object removal using mask-based inputs and efficient positional encoding.Microsoft52 upvotesRead →Simplifying and Improving On-Policy Distillation for Few-Step DiffusionThis paper introduces a simplified regression-based method for distilling diffusion models that improves generation speed without requiring complex training techniques.Microsoft27 upvotesRead →MiniCorp: The Last Mile of the AI Agent FirmMiniCorp is an office simulator that allows researchers to study how agents run companies and generate enterprise data for training.Microsoft39 upvotesRead →
Other training and scaling papers
TopicScaling Laws for Neural Language ModelsLanguage model loss falls as a smooth power law as model size, data and compute grow.
OpenAI9,592 citations12 upvotesRead →Training Compute-Optimal Large Language ModelsChinchilla: for a fixed compute budget, train a smaller model on more data; parameters and tokens should grow together.
Google DeepMind4,035 citations13 upvotesRead →Megatron-LM: Training Multi-Billion Parameter Language Models Using Model ParallelismMegatron-LM splits each Transformer layer across GPUs to train models with billions of parameters.
NVIDIA3,387 citations5 upvotesRead →DeepSeek LLM: Scaling Open-Source Language Models with LongtermismDeepSeek LLM, an open-source language model project, develops a large dataset and employs SFT and DPO to achieve performance surpassing LLaMA-2 70B and GPT-3.5 in various benchmarks and open-ended evaluations.
DeepSeek872 citations56 upvotesRead →LoRA Learns Less and Forgets LessLoRA learns less than full fine-tuning on code and math, but forgets less of what the model already knew.
Databricks486 citations91 upvotesRead →jina-embeddings-v3: Multilingual Embeddings With Task LoRAjina-embeddings-v3, a large-scale text embedding model, achieves state-of-the-art performance in multilingual and long-context retrieval tasks using Low-Rank Adaptation and Matryoshka Representation Learning.
Jina AI221 citations35 upvotesRead →
About this paper
- Venue
- arXiv.org
- Citations
- 2, 0 influential · Semantic Scholar
- Upvotes
- 34 · Hugging Face
- Lab
- Microsoft · on Companies · on Acquisitions · on Quarterly · on Paydays · on Releases · on TechConf
Changes
Citations rose from 1 to 2.
A first count of 0 influential citations.
A first count of 1 citations.
Found by the weekly Hugging Face scan; not yet checked by hand.