Papersby GlyphMultimodal and robotics
Vision, video, audio, world models and robot control. 122 papers from 26 labs.
VFlash: Fast, Accurate Visual Planning via Flash Primitive DecodingA method called VFlash speeds up and improves visual motion planning for robots by predicting entire spatial shapes at once.
NVIDIA109 upvotesRead →MoviEdit: Long-form Multi-Shot Video Editing Made EasyMoviEdit is a framework and benchmark for editing long videos across multiple camera shots and changing scenes.
Adobe75 upvotesRead →Apriel Voice 27B Preview: An Open Model for Speech, Reasoning, and Tool UseApriel Voice 27B is an open AI model that directly processes spoken speech to converse, reason, and use external tools.
ServiceNow37 upvotesRead →StreamVibe: Proactive Video Agents for Evolving User IntentStreamVibe introduces a benchmark, streaming vision-language model, and hierarchical memory harness to enable proactive, intent-aware interaction with implicit and evolving user goals.
Google78 upvotesRead →InteractiveWorld: A Verifiable Human-Centric Environment for Building Better Interactive MLLM AssistantsInteractiveWorld provides a verifiable simulation environment and benchmark to evaluate whether multimodal language models can reliably guide humans through physical tasks.
Meta46 upvotesRead →Qwen-Image-PE: Learning Visual Design for Text-to-Image Prompt EnhancementQwen-Image-PE improves text-to-image prompts via two-stage training with caption-based reverse distillation and groupwise design rewards, scaling across data, model size, and information density.
Alibaba (Qwen)23 upvotesRead →World-in-Context: Interactive Video World Modeling with In-Context ConditioningWorld-in-Context unifies view, action, and camera controls via explicit tokens and interaction-aware positional encoding to improve interactive video generation and navigation.Alibaba (Qwen)22 upvotesRead →Beyond Watching and Searching: Benchmarking Agents for Video-centered Deep Research ReportsThis benchmark evaluates how well AI agents can research and synthesize information from videos to produce interactive reports.Alibaba (Qwen)19 upvotesRead →Joints, Camera, Action! Co-Generating 3D Motion and Video for Arbitrary RigsThis model generates both video and 3D motion for any character rig, allowing the motion to be replayed accurately within a 3D scene.NVIDIA26 upvotesRead →OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal MemoryThis framework improves multimodal AI reasoning by providing persistent memory and tool interfaces to help models process and combine complex visual evidence.
Apple101 upvotesRead →BlenderWorld: Benchmarking Visual Agents for 3D Creation from Real-World ReferencesThis benchmark evaluates how well AI models can perform complex 3D creation and editing tasks within the Blender software.Alibaba (Qwen)23 upvotesRead →AesCode: Aesthetic Code Generation with Decoupled Cross-Modal RewardsThis paper introduces a method to improve code-based visual generation by using image-based aesthetic guidance and decoupled rewards to ensure quality and correctness.
Microsoft137 upvotesRead →Fish Audio Drama 3 Technical ReportThis paper introduces an open-source speech generation model capable of multi-turn performance, character-specific voice cloning, and real-time audio editing.
Fish Audio26 upvotesRead →LensErase: A Foundation Model for Erase AnythingThis paper introduces a foundation model designed for versatile image editing and object removal using mask-based inputs and efficient positional encoding.Microsoft52 upvotesRead →Urbex: Agentic Spatial Grounding for City-Scale 3D ScenesThis paper presents an agentic framework that treats city-scale 3D scene grounding as an active search task rather than a simple candidate ranking problem.Alibaba (Qwen)134 upvotesRead →Long-WAM: Scaling the Context of World-Action ModelsLong-WAM improves robot control by using autoregressive pretraining to effectively leverage long visual histories for real-time action prediction.NVIDIA2 citations114 upvotesRead →VIS-Ground: Video Interactive Storytelling with Contextual GroundingThis paper introduces a framework that transforms heterogeneous user inputs into structured constraints to improve grounding in interactive video generation.Google103 upvotesRead →Native Action-Prior Learning from Videos for World Action ModelsNAVA-WAM learns robot action priors directly from observation-only videos, improving control performance without needing action-labeled data for pretraining.Meta88 upvotesRead →World Action Modeling with Progressive Visual PlanningProWAM improves robotic control by predicting sparse visual sub-goals to guide action generation, avoiding the need for dense video generation.Meta1 citations98 upvotesRead →OmniReasoning: Pushing the Limits of Audio-Visual Joint ReasoningOmniReasoning improves audio-visual joint reasoning through a new benchmark, a data engine, and a self-distillation method for credit assignment.Alibaba (Qwen)27 upvotesRead →LongLive-Plug: Once-for-All Distillation for Video GenerationLongLive-Plug distills reusable capabilities into LoRAs that can be deployed on diverse video generation models without retraining.NVIDIA1 citations42 upvotesRead →OmniTaskonomy: When Does Visual Generation Improve Visual Understanding?This paper maps how visual generation tasks improve visual understanding capabilities, revealing specific task-dependent transfer benefits.
UC Berkeley57 upvotesRead →PixelUMM: Encoder-Free Unified Image and Video Understanding and GenerationPixelUMM is an encoder-free model that unifies image and video understanding and generation directly in pixel space.NVIDIA2 citations33 upvotesRead →LEGO-Anything: Coding Agents for 3D Scene ReconstructionLEGO-Anything is a framework where an agent iteratively writes and executes code to reconstruct 3D scenes from images.
AWS135 upvotesRead →Training Object Permanence in World ModelsThis paper introduces a dataset and training stack to improve object permanence and physical reasoning in video world models.
Carnegie Mellon University242 upvotesRead →MintAct: A Unified Visual Agent for Digital EnvironmentsMintAct is a unified vision-language model family that handles UI grounding and multi-step navigation across various digital environments.Apple35 upvotesRead →OmniVChat: Synthesizing, Benchmarking, and Training for Native Audio-Visual DialogueOmniVChat is a framework for native audio-visual dialogue that includes a data engine for synthesis and a reinforcement learning reward design.Alibaba (Qwen)1 citations149 upvotesRead →FAMOS: Feed-Forward 3D Articulation Modeling from Sparse ObservationsFAMOS is a feed-forward model that predicts articulated object motion from sparse point cloud observations.
Stanford University40 upvotesRead →SteerDuplex: Steerable Duplex Speech Dialogue ModelsA full-duplex speech model that follows spoken instructions on tone, persona, pace and voice, with a benchmark of 390 prompts to test it.
Scale AI3 citations6 upvotesRead →Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous DrivingQwen-Drive-1.0 is a vision-language foundation model for autonomous driving that unifies 3D perception, visual question answering, and motion planning via shared representations and staged training.Alibaba (Qwen)18 citations313 upvotesRead →DreamX-Phi 1.0: Action-Conditioned Video World Model for Robotic ManipulationDreamX-Phi 1.0 is an action-conditioned video world model for robotic manipulation that uses geometric attention encoding, depth estimation, object masks with a frozen teacher, and distillation to generate faithful future observations.Alibaba (Qwen)8 citations61 upvotesRead →Beyond Starry Night: Shortcut-Aware Control-State Planning for Artist-Grounded Text to Image GenerationAtelier improves artist-grounded image generation by translating vague artistic intent into explicit control states that separate scene content from style, reducing reliance on stereotypical shortcuts.Adobe1 citations28 upvotesRead →Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and RecipesControlled experiments on when text, image understanding and image generation help or compete when trained together.Meta7 citations63 upvotesRead →HumanCLAW: Can Vision-Language Models Act Through a Body?HumanCLAW decouples high-level vision-language decisions from low-level motor execution to evaluate embodied action intelligence, revealing that current vision-language models lack embodied self-awareness.Meta4 citations75 upvotesRead →Concurrent Image Understanding and Generation: Self-Correcting Coupled Markov Jump ProcessesA coupled Markov jump process with cross-modal attention and remasking enables a training-free single-pass sampler for joint multimodal generation that improves with more denoising steps.Google1 citations29 upvotesRead →Evidence-Backed Video Question AnsweringEvidence-Backed Video Question Answering requires models to provide answers with precise spatio-temporal segmentation evidence, revealing gaps between reasoning and visual grounding that are improved by large-scale instruction tuning.
Salesforce2 citations4 upvotesRead →ResearchStudio-Reel: Automate the Last Mile of Research from Paper to Poster, Video, and BlogResearchStudio-Reel automates research dissemination by composing specialized skills around a shared paper extractor, generating consistent and editable artifacts including posters, videos, and blogs with hard pass/fail quality gates.Microsoft2 citations60 upvotesRead →PhysisForcing: Physics Reinforced World Simulator for Robotic ManipulationPhysisForcing enhances embodied video generation by enforcing physical consistency through pixel-level trajectory alignment and semantic-level relational alignment losses in a DiT-based framework.NVIDIA9 citations53 upvotesRead →Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World ModelsAutoregressive video diffusion extends diffusion distillation frameworks to real-time streaming generation through causal training paradigms, achieving state-of-the-art performance with fast convergence and interactive world modeling capabilities.NVIDIA37 citations28 upvotesRead →FLAT: Feedforward Latent Triangle Splatting for Geometrically Accurate Scene GenerationVideo diffusion models are adapted to decode explicit surface primitives directly from latent space, enabling high-quality 3D scene generation with improved geometric accuracy and real-time rendering capabilities.Google1 citations26 upvotesRead →MolmoMotion: Forecasting Point Trajectories in 3D with Language Instruction3D point motion forecasting model predicts object trajectories from visual history and language goals, demonstrating superior performance on benchmarks and transferring effectively to robot manipulation and video generation tasks.
Ai216 citations53 upvotesRead →Memento: Reconstruct to Remember for Consistent Long Video GenerationMemento is a subject-reconstruction-guided framework that improves long-form video generation by preserving recurring subjects through memory-based reconstruction and dual-query mechanisms.
Baidu6 citations18 upvotesRead →Latent Spatial Memory for Video World ModelsLatent spatial memory for video world models stores 3D scene information directly in diffusion latent space, eliminating pixel-space reconstruction overhead and achieving faster generation with reduced memory usage.Microsoft24 citations71 upvotesRead →Qwen-Image-Flash: Beyond Objective DesignFew-step distillation for visual generative models benefits from systematic investigation of training recipes beyond just distillation objectives, leading to improved student performance through optimized data composition, teacher guidance, and task mixture.Alibaba (Qwen)38 upvotesRead →Cosmos 3: Omnimodal World Models for Physical AIOne model that reads and generates text, images, video, audio and robot actions for physical AI.NVIDIA225 citations144 upvotesRead →Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot EmbodimentsA unified vision-language-action model is presented that integrates diverse embodied decision-making tasks through a shared architecture and training approach, demonstrating strong performance across manipulation, navigation, and trajectory prediction with generalization across different robot platforms and environments.Alibaba (Qwen)65 citations145 upvotesRead →SANA-Streaming: Real-time Streaming Video Editing with Hybrid Diffusion TransformerSANA-Streaming enables real-time high-resolution video-to-video editing through a hybrid diffusion transformer architecture, cycle-reverse regularization, and efficient system co-design optimized for consumer GPUs.NVIDIA15 citations38 upvotesRead →Why Far Looks Up: Probing Spatial Representation in Vision-Language ModelsVision-language models exhibit entangled spatial representations that correlate vertical image position with distance, impacting reasoning robustness and performance across benchmarks.NVIDIA3 citations53 upvotesRead →LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box DecodingParallel Box Decoding enables efficient and accurate unified visual grounding and detection by decoding geometric elements as atomic units, improving both throughput and localization quality.NVIDIA38 citations142 upvotesRead →Sensor2Sensor: Cross-Embodiment Sensor Conversion for Autonomous DrivingSensor2Sensor generates high-fidelity multi-modal sensor data from in-the-wild dashcam videos using diffusion models and 4D Gaussian Splatting for autonomous driving system training and validation.Google3 citations25 upvotesRead →
AWS135 upvotesRead →Training Object Permanence in World ModelsThis paper introduces a dataset and training stack to improve object permanence and physical reasoning in video world models.