Skip to content
Papers.

OasisKV: Scaling In-Decode KV Cache Beyond HBM with Lookahead Sparse Prefetching research paper by Microsoft, 2026

Microsoft · Aug 8, 2026 · Inference and efficiency · 0 citations · 25 upvotes · unverified

Read on arXiv

What it shows

OasisKV improves LLM inference throughput by storing full KV caches in lower memory tiers and prefetching only relevant entries into HBM using speculative-decoding lookahead predictions.

UnverifiedHugging Face's summary; not yet checked by hand.

More from Microsoft

All 45
PaperCitations
The Tasteful Agent: Measuring and Improving Taste in Long-Horizon TasksLLM agents increasingly work on long-horizon tasks, and the decisions they make along the way, such as which hypothesis to test or which implementation to build on, determine the outcome of the whole run.Agents and evaluation · Sep 2026 · Unverified0
When EOS Tokens Disagree: Understanding Length Inflation in On-Policy DistillationWe study length inflation in on-policy distillation (OPD), where student responses can become excessively long and even exhaust the generation budget.Foundation models · Sep 2026 · Unverified0
When2Think: Learning Difficulty-Aware Length Control for Efficient Hybrid Reasoning ModelsLarge Reasoning Models (LRMs) achieve strong performance on complex tasks but exhibit systematic inefficiency: they often overthink easy problems and underthink hard ones.Reasoning · Sep 2026 · Unverified0
BI-Agent and BI-Bench: Towards Automating End-to-End Business IntelligenceBusiness intelligence (BI) is a cornerstone of enterprise decision-making and is widely used by enterprise users in software such as Power BI and Tableau.Agents and evaluation · Sep 2026 · Unverified0
StudentSim: Training LLM-based Student SimulatorsStudentSim trains per-student simulators that answer like a given learner and change their answers under a tutor's guidance.Applied AI · Sep 20260
AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution TracesAutoSaddler automatically improves LLM agent harnesses via offline failure-driven optimization, boosting performance on long-horizon benchmarks.Agents and evaluation · Aug 2026 · Unverified2
Agent Lightning v1.0: Towards Harnessed Agentic RLAgent Lightning v1.0 enables reproducible reinforcement learning for arbitrary agent harnesses, substantially improving coding-agent performance with minimal data and compute.Agents and evaluation · Aug 2026 · Unverified2
Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation ModelMage-VL is a codec-native streaming vision-language model that uses motion-aware tokenization and a dual-system architecture to enable efficient real-time multimodal perception and reasoning.Inference and efficiency · Jul 2026 · Unverified3

Other inference and efficiency papers

Topic
PaperCitations
Efficient Memory Management for Large Language Model Serving with PagedAttentionvLLM manages the KV cache like pages of virtual memory, serving models with 2 to 4 times the throughput.UC Berkeley · Sep 20238,481
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-AwarenessFlashAttention computes exact attention with far fewer GPU memory reads and writes, making long sequences faster.Stanford University · May 20225,334
Group Sequence Policy OptimizationGroup Sequence Policy Optimization (GSPO) is a reinforcement learning algorithm that improves training efficiency and performance of large language models by using sequence-level importance ratios and operations.Alibaba (Qwen) · Jul 2025 · Unverified688
Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse AttentionNSA, a trainable sparse attention mechanism, enhances long-context modeling efficiency without sacrificing performance, achieving improvements in speed and accuracy over full attention models.DeepSeek · Feb 2025 · Unverified507
SmolVLM: Redefining small and efficient multimodal modelsSmolVLM, a series of compact multimodal models, achieves high performance with minimal GPU memory usage, making efficient deployment on mobile and edge devices possible.Hugging Face · Apr 2025 · Unverified294
Inference-Time Scaling for Generalist Reward ModelingSelf-Principled Critique Tuning enhances pointwise generative reward modeling for large language models, improving scalability and quality compared to existing methods.DeepSeek · Apr 2025 · Unverified249
About this paper
Authors
Can Xiao, Sukmin Cho, Junbong We and 7 more
arXiv
2608.08097 · PDF
Citations
0, 0 influential · Semantic Scholar
Upvotes
25 · Hugging Face
Lab
Microsoft · on Companies · on Acquisitions · on Quarterly · on Paydays · on Releases · on TechConf

Changes

What changed
Influential citationsfirst count: 0Sep 25, 2026
Citationsfirst count: 0Sep 25, 2026
New paperFound by the weekly scan, unverifiedSep 25, 2026

Sources: each lab's own papers and arXiv, with citation and upvote counts from Semantic Scholar and Hugging Face. One-line summaries are for orientation, not a substitute for the paper. Logos via logo.dev; trademarks belong to their owners.

New papers by email

Monday afternoons, only in weeks with new papers from the labs.

Double opt-in. Unsubscribe any time.