Notable data and AI papers

Papers627Labs45

627 papers · 45 labs

mHC: Manifold-Constrained Hyper-Connections research paper by DeepSeek, 2025

DeepSeek · Training and scaling · Dec 31, 2025 · 108 citations · 334 upvotes · unverified

Read on arXiv

What it shows

Manifold-Constrained Hyper-Connections (mHC) stabilize and scale residual connection architectures by restoring identity mapping properties through manifold projection and infrastructure optimization.

By Zhenda Xie, Yixuan Wei, Huanqi Cao and 16 more · arXiv 2512.24880 · PDF

UnverifiedHugging Face's summary; not yet checked by hand.

More from DeepSeek

All 22
Inference and efficiencySep 17, 2026DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache CompressionA 552B mixture-of-experts model with a 1M-token context, built to shrink the KV cache for agent workloads.DeepSeek158 citations188 upvotesRead →Inference and efficiencyUnverifiedJul 6, 2026DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive GenerationDSpark enhances LLM inference speed by combining parallel draft generation with adaptive verification that reduces waste and improves throughput in high-concurrency settings.DeepSeek77 citations49 upvotesRead →Agents and evaluationUnverifiedFeb 25, 2026DualPath: Breaking the Storage Bandwidth Bottleneck in Agentic LLM InferenceDualPath addresses KV-cache storage I/O bottlenecks in multi-turn LLM inference by introducing dual-path loading and dynamic load balancing across prefill and decode engines.DeepSeek22 citations55 upvotesRead →Multimodal and roboticsUnverifiedJan 28, 2026DeepSeek-OCR 2: Visual Causal FlowDeepSeek-OCR 2 introduces DeepEncoder V2 that dynamically reorders visual tokens based on semantic content, enabling more human-like causal reasoning in 2D image understanding through cascaded 1D causal structures.DeepSeek79 citations73 upvotesRead →Agents and evaluationUnverifiedDec 2, 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language ModelsDeepSeek-V3.2 introduces DeepSeek Sparse Attention and a scalable reinforcement learning framework, achieving superior reasoning and performance compared to GPT-5 and Gemini-3.0-Pro in complex reasoning tasks.DeepSeek887 citations275 upvotesRead →ReasoningUnverifiedNov 27, 2025DeepSeekMath-V2: Towards Self-Verifiable Mathematical ReasoningA self-verifying large language model for theorem proving improves mathematical reasoning by incentivizing rigorous step-by-step derivations and achieves high scores in international competitions.DeepSeek83 citations98 upvotesRead →Inference and efficiencyUnverifiedOct 21, 2025DeepSeek-OCR: Contexts Optical CompressionDeepSeek-OCR uses optical 2D mapping to compress long contexts, achieving high OCR precision with reduced vision tokens and demonstrating practical value in document processing.DeepSeek200 citations98 upvotesRead →Inference and efficiencyUnverifiedMay 14, 2025Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI ArchitecturesDeepSeek-V3 addresses hardware limitations through MLA, MoE, FP8 training, and Multi-Plane Network Topology, enabling efficient large-scale LLM training and inference.DeepSeek116 citations77 upvotesRead →

Other training and scaling papers

Topic
Training and scalingJun 17, 2021LoRA: Low-Rank Adaptation of Large Language ModelsLoRA fine-tunes a large model by training small low-rank matrices, cutting trainable parameters by 10,000 times.Microsoft25.6k citations66 upvotesRead →Training and scalingJan 23, 2020Scaling Laws for Neural Language ModelsLanguage model loss falls as a smooth power law as model size, data and compute grow.OpenAI9,592 citations12 upvotesRead →Training and scalingMar 29, 2022Training Compute-Optimal Large Language ModelsChinchilla: for a fixed compute budget, train a smaller model on more data; parameters and tokens should grow together.Google DeepMind4,035 citations13 upvotesRead →Training and scalingSep 17, 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model ParallelismMegatron-LM splits each Transformer layer across GPUs to train models with billions of parameters.NVIDIA3,387 citations5 upvotesRead →Training and scalingMay 15, 2024LoRA Learns Less and Forgets LessLoRA learns less than full fine-tuning on code and math, but forgets less of what the model already knew.Databricks486 citations91 upvotesRead →Training and scalingUnverifiedSep 16, 2024jina-embeddings-v3: Multilingual Embeddings With Task LoRAjina-embeddings-v3, a large-scale text embedding model, achieves state-of-the-art performance in multilingual and long-context retrieval tasks using Low-Rank Adaptation and Matryoshka Representation Learning.Jina AI221 citations35 upvotesRead →
About this paper
Venue
arXiv.org
Citations
108, 23 influential · Semantic Scholar
Upvotes
334 · Hugging Face
Lab
DeepSeek · on Companies · on breaches · on Rounds · on labs · on Leaders · on Arena · on legislations · on Paydays · on Metered · on problems · on repos · on socials · on standards · on terms

Changes

CitationsNov 9, 2026Influential citations rose from 22 to 23.
CitationsNov 9, 2026Citations rose from 103 to 108.
CitationsNov 2, 2026Influential citations rose from 20 to 22.
CitationsNov 2, 2026Citations rose from 93 to 103.
CitationsOct 12, 2026Citations rose from 91 to 93.
CitationsOct 5, 2026Influential citations rose from 19 to 20.
CitationsOct 5, 2026Citations rose from 83 to 91.
CitationsSep 28, 2026Citations rose from 82 to 83.
CitationsSep 25, 2026A first count of 19 influential citations.
CitationsSep 25, 2026A first count of 82 citations.
New paperSep 25, 2026Found by the weekly Hugging Face scan; not yet checked by hand.