Papersby Glyph

Notable data and AI papers

Papers627 Labs45

627 papers · 45 labs

LoRA: Low-Rank Adaptation of Large Language Models research paper by Microsoft, 2021

Microsoft · Training and scaling · Jun 17, 2021 · 25,555 citations · 66 upvotes

LoRA fine-tunes a large model by training small low-rank matrices, cutting trainable parameters by 10,000 times.

By Edward J. Hu, Yelong Shen, Phillip Wallis and 5 more · arXiv 2106.09685 · PDF

Summarised by hand from the abstract.

More from Microsoft

Agents and evaluationUnverifiedNov 2, 2026WebHarbor: Docking Real Websites for Evolving GUI Agent EnvironmentsWebHarbor provides realistic, containerized website environments for training and testing computer-using web agents across complex online tasks.

Microsoft28 upvotesRead →Agents and evaluationUnverifiedOct 19, 2026Agent World Model in the Wild: Computer-Use Agent Environment SynthesisThis pipeline creates controllable, open-source environments to safely train and test AI agents on complex, long-horizon computer-use tasks.

Microsoft69 upvotesRead →Training and scalingUnverifiedOct 16, 2026Self-Evolving Coding Agents via Harness-Policy OptimizationThis framework improves coding agents by recursively alternating between optimizing the agent's policy and the environment harness it uses to interact with code.

Microsoft48 upvotesRead →Multimodal and roboticsUnverifiedOct 13, 2026AesCode: Aesthetic Code Generation with Decoupled Cross-Modal RewardsThis paper introduces a method to improve code-based visual generation by using image-based aesthetic guidance and decoupled rewards to ensure quality and correctness.

Microsoft137 upvotesRead →Training and scalingUnverifiedOct 13, 2026APT: Feedback-Driven Agentic Post-Training for Text-to-Image GenerationThis paper introduces an automated post-training framework that uses a feedback loop of evaluation and diagnosis to improve text-to-image models.

Microsoft23 upvotesRead →Multimodal and roboticsUnverifiedOct 13, 2026LensErase: A Foundation Model for Erase AnythingThis paper introduces a foundation model designed for versatile image editing and object removal using mask-based inputs and efficient positional encoding.

Microsoft52 upvotesRead →Inference and efficiencyUnverifiedOct 13, 2026Simplifying and Improving On-Policy Distillation for Few-Step DiffusionThis paper introduces a simplified regression-based method for distilling diffusion models that improves generation speed without requiring complex training techniques.

Microsoft27 upvotesRead →Applied AIUnverifiedOct 5, 2026MiniCorp: The Last Mile of the AI Agent FirmMiniCorp is an office simulator that allows researchers to study how agents run companies and generate enterprise data for training.

Microsoft39 upvotesRead →

Training and scalingJan 23, 2020Scaling Laws for Neural Language ModelsLanguage model loss falls as a smooth power law as model size, data and compute grow.

OpenAI9,592 citations12 upvotesRead →Training and scalingMar 29, 2022Training Compute-Optimal Large Language ModelsChinchilla: for a fixed compute budget, train a smaller model on more data; parameters and tokens should grow together.

Google DeepMind4,035 citations13 upvotesRead →Training and scalingSep 17, 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model ParallelismMegatron-LM splits each Transformer layer across GPUs to train models with billions of parameters.

NVIDIA3,387 citations5 upvotesRead →Training and scalingUnverifiedJan 5, 2024DeepSeek LLM: Scaling Open-Source Language Models with LongtermismDeepSeek LLM, an open-source language model project, develops a large dataset and employs SFT and DPO to achieve performance surpassing LLaMA-2 70B and GPT-3.5 in various benchmarks and open-ended evaluations.

DeepSeek872 citations56 upvotesRead →Training and scalingMay 15, 2024LoRA Learns Less and Forgets LessLoRA learns less than full fine-tuning on code and math, but forgets less of what the model already knew.

Databricks486 citations91 upvotesRead →Training and scalingUnverifiedSep 16, 2024jina-embeddings-v3: Multilingual Embeddings With Task LoRAjina-embeddings-v3, a large-scale text embedding model, achieves state-of-the-art performance in multilingual and long-context retrieval tasks using Low-Rank Adaptation and Matryoshka Representation Learning.

Jina AI221 citations35 upvotesRead →

About this paper

Venue: International Conference on Learning Representations
Citations: 25,555, 3,547 influential · Semantic Scholar
Upvotes: 66 · Hugging Face
Lab: Microsoft · on Companies · on Acquisitions · on Quarterly · on Paydays · on Releases · on TechConf

Changes

CitationsNov 9, 2026Influential citations rose from 3,500 to 3,547.

CitationsNov 9, 2026Citations rose from 25,222 to 25,555.

CitationsNov 2, 2026Influential citations rose from 3,362 to 3,500.

CitationsNov 2, 2026Citations rose from 24,206 to 25,222.

CitationsOct 12, 2026Influential citations rose from 3,299 to 3,362.

CitationsOct 12, 2026Citations rose from 23,799 to 24,206.

CitationsOct 5, 2026Influential citations rose from 3,241 to 3,299.

CitationsOct 5, 2026Citations rose from 23,349 to 23,799.

CitationsSep 28, 2026Influential citations rose from 3,227 to 3,241.

CitationsSep 28, 2026Citations rose from 23,255 to 23,349.

CitationsSep 25, 2026Influential citations rose from 3,222 to 3,227.

CitationsSep 25, 2026Citations rose from 23,235 to 23,255.

New paperSep 24, 2026Added to the list.