Papersby GlyphReasoning
Step-by-step thinking, reinforcement learning for reasoning, math and logic. 81 papers from 24 labs.
ThinkWorld: World Imagination Reinforces Chain-of-Dynamics ReasoningThinkWorld improves robotic generalization by representing actions as dynamics chains and using world imagination to learn from unlabeled human videos.
NVIDIA57 upvotesRead →Hybrid Reasoning AgentsThis paper introduces an agent architecture that dynamically switches between latent reasoning and text-based interaction to improve performance and inference speed.
Stanford University41 upvotesRead →Thinking Inertia: LLMs Keep Thinking When Told Not ToThis paper shows that explicit controls often fail to stop LLMs from generating internal reasoning traces, a phenomenon the authors call thinking inertia.Stanford University244 upvotesRead →Self-Retrospection Distillation: Turning Post-hoc Experiences into Prior ForesightSelf-Retrospection Distillation improves agent performance by using hindsight from completed trajectories to supervise foresight predictions during training.
Salesforce1 citations197 upvotesRead →Beyond Memory: Harnessing Long-Horizon Agents with Explicit Belief StatesPoS improves long-horizon agent performance by maintaining explicit belief states that track world status and unresolved task requirements.
Alibaba (Qwen)1 citations91 upvotesRead →Selection-Based Structured Reasoning: Toward Efficient Multimodal Search AgentsSSR improves multimodal agent efficiency by replacing free-form reasoning generation with a selection process from pre-specified reasoning candidates.
Carnegie Mellon University37 upvotesRead →Asking for What Was Never Requested: Horizontal and Vertical Proactivity in AgentsQ&D trains agents to proactively ask for information that is necessary for task completion but not explicitly requested by the user.
IBM41 upvotesRead →An RL View of OPD: Least Square Policy Distillation for Sample-Efficient LLM ReasoningLSPD applies reinforcement learning principles to policy distillation, improving sample efficiency and policy diversity in language model training.NVIDIA26 upvotesRead →Rethinking Latent Visual Reasoning: Grounding Latent Reasoning in Visual EvidenceReaLVR improves latent visual reasoning by providing explicit supervision to latent trajectories, helping models focus on relevant visual evidence.
Amazon2 citations210 upvotesRead →When2Think: Learning Difficulty-Aware Length Control for Efficient Hybrid Reasoning ModelsWhen2Think improves reasoning efficiency by dynamically allocating computation based on problem difficulty, using less compute for easy tasks.
Microsoft1 citations54 upvotesRead →Beyond Solver Verdicts: Generative Reward Models for AutoformalizationNeurosymbolic reasoning is vulnerable to incorrect but verdict-matching formal translations, which are addressed by a generative verification method that scores reference equivalence without an oracle and improves downstream accuracy.
AWS1 citations33 upvotesRead →Building Multilingual Bridges: Data Mixing as the Pillar of Generalization for In-Language ReasoningOptimized supervised fine-tuning data composition enables reasoning models to consistently process and respond in diverse non-English languages without requiring reasoning supervision in each target language.
Cohere33 upvotesRead →CORE: Improving Compositional Reasoning in MLLM Embedding via Reranker DistillationCORE distills compositional ranking judgments from a cross-attentive reranker into an embedding model via synthesized multi-level candidates and a Rank-KL objective, improving compositional retrieval without degrading standard performance.Alibaba (Qwen)26 upvotesRead →Random Attention: Rethinking KV Cache Eviction for Efficient ReasoningRandom eviction of reasoning tokens matches selective KV cache compression because reasoning traces are self-protecting through redundancy, making scoring unnecessary once prompts are preserved.Salesforce5 citations187 upvotesRead →The IOL-AI Challenge: An Open Challenge towards Advancing Linguistic ReasoningAn open competition on unseen Linguistics Olympiad puzzles, graded by the official jury; small 14B systems beat models twice their size.Cohere1 citationsRead →CALIBER: Calibrating Confidence Before and After Reasoning in Language ModelsTrains reasoning models to state their confidence twice, before thinking and after answering, halving calibration error against the best single-estimate method.Cohere1 citationsRead →Learning from Language Feedback via Variational Policy DistillationVariational Policy Distillation enables reinforcement learning from language feedback by co-evolving teacher and student policies through variational expectation-maximization, overcoming limitations of passive distillation in complex reasoning tasks.Salesforce7 citations12 upvotesRead →Reward Hacking in Rubric-Based Reinforcement LearningResearch examines reward hacking in rubric-based reinforcement learning, identifying verifier failure and rubric-design limitations as key sources of divergence between training and evaluation metrics.
Scale AI33 citations4 upvotesRead →Stabilizing Efficient Reasoning with Step-Level Advantage SelectionShort-context post-training induces reasoning compression but causes instability; Step-level Advantage Selection addresses this by selectively adjusting reasoning steps based on confidence and verification outcomes, improving accuracy-efficiency trade-off in reasoning tasks.
AMD2 citations8 upvotesRead →Maximal Brain Damage Without Data or Optimization: Disrupting Neural Networks via Sign-Bit FlipsDeep neural networks exhibit catastrophic vulnerability to minimal parameter bit flips across multiple domains, which can be identified and mitigated through targeted protection strategies.NVIDIA4 citations57 upvotesRead →Think in Strokes, Not Pixels: Process-Driven Image Generation via Interleaved ReasoningProcess-driven image generation decomposes synthesis into iterative steps involving textual planning, visual drafting, textual reflection, and visual refinement, with step-wise supervision ensuring consistency and interpretability.
Meta6 citations71 upvotesRead →Graph-Based Chain-of-Thought Pruning for Reducing Redundant Reflections in Reasoning LLMsA graph-based framework optimizes chain-of-thought reasoning in large language models by identifying and eliminating redundant thinking patterns through structured pruning and reinforcement learning techniques.
Baidu2 citations12 upvotesRead →TriAttention: Efficient Long Reasoning with Trigonometric KV CompressionTriAttention addresses KV cache memory bottlenecks in LLMs by leveraging Q/K vector concentration in pre-RoPE space to improve key importance estimation and enable efficient long-context generation.NVIDIA25 citations112 upvotesRead →Apriel-Reasoner: RL Post-Training for General-Purpose and Efficient ReasoningApriel-Reasoner is a 15B-parameter language model trained with reproducible multi-domain reinforcement learning to improve reasoning efficiency and accuracy across diverse tasks while reducing inference costs.
ServiceNow12 upvotesRead →Therefore I am. I ThinkReasoning models appear to encode action choices before beginning textual deliberation, as evidenced by early decision detection and activation steering effects.ServiceNow2 citations31 upvotesRead →Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?Self-distillation in large language models can degrade mathematical reasoning performance by suppressing uncertainty expression, particularly affecting out-of-distribution tasks.Microsoft110 citations57 upvotesRead →On the Direction of RLVR Updates for LLM Reasoning: Identification and ExploitationReinforcement learning with verifiable rewards improves language model reasoning by focusing on the direction of parameter updates rather than their magnitude, enabling better test-time extrapolation and training-time reweighting methods.Alibaba (Qwen)16 citations29 upvotesRead →FIPO: Eliciting Deep Reasoning with Future-KL Influenced Policy OptimizationFIPO enhances reinforcement learning for language models by using discounted future-KL divergence to improve credit assignment and extend reasoning chains, achieving better mathematical problem-solving performance.Alibaba (Qwen)31 citations104 upvotesRead →HopChain: Multi-Hop Data Synthesis for Generalizable Vision-Language ReasoningHopChain is a scalable framework that generates multi-hop vision-language reasoning data to enhance VLMs' long-chain reasoning capabilities across diverse benchmarks.Alibaba (Qwen)7 citations65 upvotesRead →Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMsReasoning in large language models enhances parametric knowledge recall through computational buffer and factual priming mechanisms, though it carries risks of hallucination that can be mitigated by prioritizing accurate reasoning trajectories.
Google17 citations76 upvotesRead →Reasoning Models Struggle to Control their Chains of ThoughtChain-of-thought controllability measures how effectively models can be constrained to follow reasoning steps, with findings showing significantly lower controllability in reasoning versus output generation, and varying impacts from model size, training methods, and task complexity.
OpenAI37 citations41 upvotesRead →Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience MemoryA memory mechanism called Memex enables large language model agents to handle long-horizon tasks more effectively by maintaining compact context through structured summaries while storing full interaction details in an external database, allowing selective retrieval based on learned criteria.
Accenture17 citations19 upvotesRead →LoGeR: Long-Context Geometric Reconstruction with Hybrid MemoryLoGeR enables long-term 3D video reconstruction by combining bidirectional priors with a hybrid memory system that includes parametric Test-Time Training and non-parametric sliding window attention mechanisms.
Google DeepMind46 citations63 upvotesRead →CHIMERA: Compact Synthetic Data for Generalizable LLM ReasoningA synthetic reasoning dataset called CHIMERA is introduced to overcome data-centric challenges in training large language models for cross-domain reasoning, achieving performance comparable to much larger models.
Apple2 citations57 upvotesRead →Unlocking Reasoning Capability on Machine Translation in Large Language ModelsFinds that step-by-step reasoning makes machine translation worse, then trains a model on structured draft-and-revise traces that improve it.Cohere2 citationsRead →Improving Data and Reward Design for Scientific Reasoning in Large Language ModelsA large-scale scientific question dataset and post-training pipeline are developed to improve open-ended science question answering through enhanced data processing and reinforcement learning with rubric-guided evaluation.Microsoft5 citations44 upvotesRead →Recurrent-Depth VLA: Implicit Test-Time Compute Scaling of Vision-Language-Action Models via Latent Iterative ReasoningRD-VLA introduces a recurrent architecture for vision-language-action models that adapts computational depth through latent iterative refinement, achieving constant memory usage and improved task success rates.
Ai221 citations71 upvotesRead →Reinforced Attention LearningReinforced Attention Learning optimizes internal attention distributions in multimodal language models, improving information allocation and cross-modal alignment through policy-gradient methods.Google8 citations30 upvotesRead →Golden Goose: A Simple Trick to Synthesize Unlimited RLVR Tasks from Unverifiable Internet TextGolden Goose synthesizes unlimited RLVR tasks from unverifiable internet text by creating multiple-choice question-answering versions of fill-in-the-middle tasks, enabling large-scale training and achieving state-of-the-art results in cybersecurity and other domains.NVIDIA12 citations113 upvotesRead →Less Noise, More Voice: Reinforcement Learning for Reasoning via Instruction PurificationLENS framework improves reinforcement learning with verifiable rewards by identifying and removing interference tokens to enhance exploration efficiency and training stability.Baidu12 upvotesRead →Teaching Models to Teach Themselves: Reasoning at the Edge of LearnabilityA self-improvement framework enables pretrained language models to generate automated curricula for solving previously unsolvable problems by leveraging latent knowledge and meta-reinforcement learning.Meta25 citations43 upvotesRead →Fast-ThinkAct: Efficient Vision-Language-Action Reasoning via Verbalizable Latent PlanningFast-ThinkAct is an efficient vision-language-action framework that reduces inference latency by 89.3% through compact latent reasoning while maintaining long-horizon planning and few-shot adaptation capabilities.NVIDIA31 citations54 upvotesRead →Ministral 3The Ministral 3 series consists of parameter-efficient dense language models with three sizes (3B, 8B, 14B) and three variants per size, trained using cascade distillation for compute-constrained applications.
Mistral AI80 citations66 upvotesRead →Multiplex Thinking: Reasoning via Token-wise Branch-and-MergeMultiplex Thinking introduces a stochastic soft reasoning mechanism that samples multiple candidate tokens at each step to optimize reasoning trajectories with reinforcement learning while maintaining shorter sequences than traditional chain-of-thought methods.Microsoft19 citations40 upvotesRead →X-Coder: Advancing Competitive Programming with Fully Synthetic Tasks, Solutions, and TestsCode LLMs trained on fully synthetic data using a feature-based synthesis pipeline achieve superior performance on competitive programming benchmarks while reducing dependence on real-world coding datasets.Microsoft1 citations48 upvotesRead →VideoAuto-R1: Video Auto Reasoning via Thinking Once, Answering TwiceVideoAuto-R1 framework employs a reason-when-necessary strategy for video understanding, using a Thinking Once, Answering Twice training paradigm with verifiable rewards and confidence-based reasoning activation during inference.Meta23 citations38 upvotesRead →Latent Implicit Visual ReasoningThis paper introduces a task-agnostic mechanism for models to discover and use visual reasoning tokens without explicit supervision.
UC Berkeley19 citations60 upvotesRead →Emergent temporal abstractions in autoregressive models enable hierarchical reinforcement learningThis paper shows that autoregressive models can learn temporally-abstract actions within their internal representations to enable efficient hierarchical reinforcement learning.Google5 citations62 upvotesRead →Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning ModelsCascaded domain-wise reinforcement learning (Cascade RL) is proposed to enhance general-purpose reasoning models, achieving state-of-the-art performance across benchmarks and outperforming the teacher model in coding competitions.NVIDIA53 citations40 upvotesRead →Olmo 3Olmo 3, a family of state-of-the-art fully-open language models at 7B and 32B parameter scales, excels in long-context reasoning, function calling, coding, instruction following, general chat, and knowledge recall.Ai237 upvotesRead →
Amazon2 citations210 upvotesRead →When2Think: Learning Difficulty-Aware Length Control for Efficient Hybrid Reasoning ModelsWhen2Think improves reasoning efficiency by dynamically allocating computation based on problem difficulty, using less compute for easy tasks.AWS1 citations33 upvotesRead →Building Multilingual Bridges: Data Mixing as the Pillar of Generalization for In-Language ReasoningOptimized supervised fine-tuning data composition enables reasoning models to consistently process and respond in diverse non-English languages without requiring reasoning supervision in each target language.