Skip to content
Papers.

PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation research paper by NVIDIA, 2026

NVIDIA · Sep 29, 2026 · Multimodal and robotics · 28 upvotes · unverified 6 days ago

Read on arXiv

What it shows

Unified Multimodal Models (UMMs) often rely on separate visual representations for understanding and generation, increasing visual context length and complicating integration with established vision-language...

By Cong Wei, Xuanchi Ren, Bryan Chu and 8 more · arXiv 2609.38597 · PDF · Code

UnverifiedHugging Face's summary; not yet checked by hand.

More from NVIDIA

All 79
PaperCitations
Mid-Harness: Scaling Actions Between Model and Harness for Terminal AgentsTerminal agents act through stochastic model generations, yet the ability to generate a useful action does not ensure its reliable execution.Agents and evaluation · Sep 2026 · Unverified5 days ago-
LongLive-Plug: Once-for-All Distillation for Video GenerationVideo diffusion models are increasingly developed into specialized models for diverse downstream tasks, and this development often includes a distillation stage, for example to accelerate sampling or to improve...Multimodal and robotics · Sep 2026 · Unverified6 days ago-
An RL View of OPD: Least Square Policy Distillation for Sample-Efficient LLM ReasoningWe study on-policy distillation (OPD) through the lens of reinforcement learning, establishing a connection between the reverse-KL objective in OPD and KL-regularized policy optimization.Reasoning · Sep 2026 · Unverified7 days ago-
SoL-Pi: Recursively Scaling Auto-Research Loops for Efficient Agent HarnessAs coding agents move from supervised code completion to unattended, around-the-clock exploration, their work expands from isolated predictions into long trajectories of reasoning, tool use, and feedback.Agents and evaluation · Sep 2026 · Unverified2 weeks ago4
An Open Recipe for IMO Gold: Training Nemotron for Olympiad MathematicsA natural-language proof-generation pipeline using post-trained Nemotron 3 Ultra checkpoints achieves gold-medal performance on IMO 2026 through iterative verification and refinement without external tools.Training and scaling · Sep 2026 · Unverified3 weeks ago0
Online Draft Co-Training for Speculative Decoding in Large-Scale, Long-Context RL Post-TrainingA system for large-scale online draft co-training accelerates speculative decoding in RL post-training by extending context-parallel attention and adding cross-stage feature transport.Training and scaling · Sep 2026 · Unverified4 weeks ago2
Sol-Attn: Accelerating Video Generation Inference via On-the-Fly Attention SparsificationSol-Attn improves training-free sparse attention for diffusion transformers by combining dynamic block routing, sparse computation, and approximation correction in a single online pass to accelerate video generation without sacrificing quality.Inference and efficiency · Jul 2026 · Unverified2 months ago6
SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video GenerationSANA-Video 2.0 is a hybrid video diffusion transformer that combines linear and softmax attention to generate high-resolution video efficiently on a single GPU.Inference and efficiency · Jul 2026 · Unverified2 months ago2

Other multimodal and robotics papers

Topic
PaperCitations
Segment AnythingA promptable model and a dataset of over a billion masks that cut out any object in any image.Meta · Apr 20233 years ago15.9k
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsBLIP-2 efficiently pre-trains vision-language models using off-the-shelf frozen encoders and decoders, achieving state-of-the-art performance with fewer parameters.Salesforce · Jan 20233 years ago9,360
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and GenerationBLIP, a Vision-Language Pre-training framework, improves performance across both understanding and generation tasks by bootstrapping captions from noisy web data, achieving state-of-the-art results on image-text retrieval, image captioning, and VQA, and showing strong generalization to video-language tasks.Salesforce · Jan 20224 years ago7,410
GPT-4o System CardGPT-4o is an omnimodal autoregressive model trained to handle text, audio, image, and video inputs, offering high-performance outputs across these modalities, with particular strengths in vision and audio.OpenAI · Oct 2024 · Unverified1 year ago5,071
SAM 3: Segment Anything with ConceptsSegment Anything Model 3 achieves state-of-the-art performance in promptable concept segmentation and tracking by leveraging a unified model architecture with decoupled recognition and localization.Meta · Nov 2025 · Unverified10 months ago1,105
DeepSeek-VL: Towards Real-World Vision-Language UnderstandingDeepSeek-VL is an open-source vision-language model that achieves state-of-the-art performance in real-world applications by combining a hybrid vision encoder with effective pretraining strategies to preserve language model capabilities.DeepSeek · Mar 2024 · Unverified2 years ago896
About this paper
Authors
Cong Wei, Xuanchi Ren, Bryan Chu and 8 more
arXiv
2609.38597 · PDF
Citations
Not counted yet · Semantic Scholar
Upvotes
28 · Hugging Face
Code
github.com/nv-tlabs/PixelUMM
Lab
NVIDIA · on Companies · on Acquisitions · on Quarterly · on Paydays · on TechConf

Changes

What changed
New paperFound by the weekly scan, unverifiedOct 5, 2026today

New papers by email

Monday afternoons, only in weeks with new papers from the labs.

Double opt-in. Unsubscribe any time.