Papersby GlyphCosmos 3: Omnimodal World Models for Physical AI research paper by NVIDIA, 2026
NVIDIA · Multimodal and robotics · Jun 1, 2026 · 225 citations · 144 upvotes
What it shows
One model that reads and generates text, images, video, audio and robot actions for physical AI.
By Aditi, Niket Agarwal, Arslan Ali and 288 more · arXiv 2606.02800 · PDF · Code
Summarised by hand from the abstract.
The model it introduced
On models.fru.devMore from NVIDIA
All 90VFlash: Fast, Accurate Visual Planning via Flash Primitive DecodingA method called VFlash speeds up and improves visual motion planning for robots by predicting entire spatial shapes at once.
NVIDIA109 upvotesRead →Nemotron-Labs-Diffusion-Uni: Elastic Self-Refining Unified Diffusion LLM with Cross-Paradigm On-Policy DistillationNLD-Uni is a unified discrete diffusion language model that combines visual understanding and generation via cross-paradigm on-policy distillation and progressive self-refining diffusion.NVIDIA12 upvotesRead →ThinkWorld: World Imagination Reinforces Chain-of-Dynamics ReasoningThinkWorld improves robotic generalization by representing actions as dynamics chains and using world imagination to learn from unlabeled human videos.NVIDIA57 upvotesRead →Joints, Camera, Action! Co-Generating 3D Motion and Video for Arbitrary RigsThis model generates both video and 3D motion for any character rig, allowing the motion to be replayed accurately within a 3D scene.NVIDIA26 upvotesRead →EdgeWise: Bringing Long-Horizon Agents to the Edge via Selective Cloud DelegationThis paper introduces a framework that reduces inference costs by using a small local model for most tasks and delegating only critical steps to the cloud.NVIDIA30 upvotesRead →When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often BetterThis paper shows that distilling from offline student rollouts is often more efficient and accurate than standard on-policy distillation for many model pairs.NVIDIA28 upvotesRead →Long-WAM: Scaling the Context of World-Action ModelsLong-WAM improves robot control by using autoregressive pretraining to effectively leverage long visual histories for real-time action prediction.NVIDIA2 citations114 upvotesRead →NeMo-DCR: Bit-Exact Delta-Compressed Refit for Scalable Agentic RL at Trillion-Parameter ScaleNeMo-DCR enables efficient, bit-exact synchronization of trillion-parameter model checkpoints by transmitting only the sparse weight changes between clusters.NVIDIA1 citations29 upvotesRead →
Other multimodal and robotics papers
TopicSegment AnythingA promptable model and a dataset of over a billion masks that cut out any object in any image.
Meta16.8k citations6 upvotesRead →BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsBLIP-2 efficiently pre-trains vision-language models using off-the-shelf frozen encoders and decoders, achieving state-of-the-art performance with fewer parameters.
Salesforce9,805 citations3 upvotesRead →BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and GenerationBLIP, a Vision-Language Pre-training framework, improves performance across both understanding and generation tasks by bootstrapping captions from noisy web data, achieving state-of-the-art results on image-text retrieval, image captioning, and VQA, and showing strong generalization to video-language tasks.Salesforce7,655 citations5 upvotesRead →GPT-4o System CardGPT-4o is an omnimodal autoregressive model trained to handle text, audio, image, and video inputs, offering high-performance outputs across these modalities, with particular strengths in vision and audio.
OpenAI5,388 citations88 upvotesRead →SAM 3: Segment Anything with ConceptsSegment Anything Model 3 achieves state-of-the-art performance in promptable concept segmentation and tracking by leveraging a unified model architecture with decoupled recognition and localization.Meta1,466 citations138 upvotesRead →DeepSeek-VL: Towards Real-World Vision-Language UnderstandingDeepSeek-VL is an open-source vision-language model that achieves state-of-the-art performance in real-world applications by combining a hybrid vision encoder with effective pretraining strategies to preserve language model capabilities.
DeepSeek939 citations49 upvotesRead →
About this paper
- Venue
- arXiv.org
- Citations
- 225, 35 influential · Semantic Scholar
- Upvotes
- 144 · Hugging Face
- Lab
- NVIDIA · on Companies · on Acquisitions · on Quarterly · on Paydays · on TechConf
Changes
Influential citations rose from 23 to 35.
Citations rose from 151 to 225.
Hugging Face upvotes rose from 142 to 144.
Influential citations rose from 16 to 23.
Citations rose from 125 to 151.
Influential citations rose from 9 to 16.
Citations rose from 94 to 125.
Citations rose from 90 to 94.
Added to the list.