Notable data and AI papers

Papers627Labs45

627 papers · 45 labs

OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal Memory research paper by Apple, 2026

Apple · Multimodal and robotics · Oct 19, 2026 · 101 upvotes · unverified

Read on arXiv

What it shows

This framework improves multimodal AI reasoning by providing persistent memory and tool interfaces to help models process and combine complex visual evidence.

By Yu Zhou, Cheng-Fu Yang, Rui Sun and 8 more · arXiv 2610.26860 · PDF · Code

UnverifiedSummarised by a model from the abstract; not yet checked by hand.

More from Apple

All 11
Inference and efficiencyUnverifiedOct 1, 2026Decoding Looped Transformers Better for (Almost) FreeLoopCD improves decoding quality in looped transformers by contrasting final predictions with earlier recurrent states without extra training.Apple1 citations46 upvotesRead →Multimodal and roboticsUnverifiedSep 18, 2026MintAct: A Unified Visual Agent for Digital EnvironmentsMintAct is a unified vision-language model family that handles UI grounding and multi-step navigation across various digital environments.Apple35 upvotesRead →Retrieval and dataUnverifiedSep 1, 2026It Takes Two to Match: Co-Evolving Generative Retriever with Reinforcement LearningCoGR trains LLMs to generate compact keywords for both queries and items, enabling direct inverted-index retrieval optimized via co-evolving reinforcement learning.Apple2 citations65 upvotesRead →Foundation modelsUnverifiedApr 1, 2026Embarrassingly Simple Self-Distillation Improves Code GenerationSimple self-distillation improves code generation in large language models by fine-tuning on model-generated samples, effectively addressing precision-exploration trade-offs in decoding.Apple40 citations55 upvotesRead →ReasoningUnverifiedMar 1, 2026CHIMERA: Compact Synthetic Data for Generalizable LLM ReasoningA synthetic reasoning dataset called CHIMERA is introduced to overcome data-centric challenges in training large language models for cross-domain reasoning, achieving performance comparable to much larger models.Apple2 citations57 upvotesRead →Multimodal and roboticsUnverifiedDec 11, 2025Sharp Monocular View Synthesis in Less Than a SecondSHARP synthesizes photorealistic views from a single image using a 3D Gaussian representation, achieving state-of-the-art results with rapid processing.Apple32 citations31 upvotesRead →Multimodal and roboticsUnverifiedDec 8, 2025One Layer Is Enough: Adapting Pretrained Visual Encoders for Image GenerationFAE, a framework using a feature auto-encoder and dual decoders, adapts pre-trained visual representations for generative models, achieving high performance in image generation tasks.Apple32 citations25 upvotesRead →Multimodal and roboticsUnverifiedNov 25, 2025STARFlow-V: End-to-End Video Generative Modeling with Normalizing FlowSTARFlow-V, a normalizing flow-based video generator, offers end-to-end learning, robust causal prediction, and high-quality video generation with practical sampling efficiency.Apple8 citations33 upvotesRead →

Other multimodal and robotics papers

Topic
Multimodal and roboticsApr 5, 2023Segment AnythingA promptable model and a dataset of over a billion masks that cut out any object in any image.Meta16.8k citations6 upvotesRead →Multimodal and roboticsJan 30, 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsBLIP-2 efficiently pre-trains vision-language models using off-the-shelf frozen encoders and decoders, achieving state-of-the-art performance with fewer parameters.Salesforce9,805 citations3 upvotesRead →Multimodal and roboticsJan 28, 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and GenerationBLIP, a Vision-Language Pre-training framework, improves performance across both understanding and generation tasks by bootstrapping captions from noisy web data, achieving state-of-the-art results on image-text retrieval, image captioning, and VQA, and showing strong generalization to video-language tasks.Salesforce7,655 citations5 upvotesRead →Multimodal and roboticsUnverifiedOct 25, 2024GPT-4o System CardGPT-4o is an omnimodal autoregressive model trained to handle text, audio, image, and video inputs, offering high-performance outputs across these modalities, with particular strengths in vision and audio.OpenAI5,388 citations88 upvotesRead →Multimodal and roboticsUnverifiedNov 20, 2025SAM 3: Segment Anything with ConceptsSegment Anything Model 3 achieves state-of-the-art performance in promptable concept segmentation and tracking by leveraging a unified model architecture with decoupled recognition and localization.Meta1,466 citations138 upvotesRead →Multimodal and roboticsUnverifiedMar 8, 2024DeepSeek-VL: Towards Real-World Vision-Language UnderstandingDeepSeek-VL is an open-source vision-language model that achieves state-of-the-art performance in real-world applications by combining a hybrid vision encoder with effective pretraining strategies to preserve language model capabilities.DeepSeek939 citations49 upvotesRead →
About this paper
Citations
Not counted yet · Semantic Scholar
Upvotes
101 · Hugging Face
Lab
Apple · on Companies · on Quarterly · on Paydays · on TechConf

Changes

UpvotesNov 9, 2026Hugging Face upvotes fell from 112 to 101.
UpvotesNov 2, 2026Hugging Face upvotes fell from 120 to 112.
New summaryOct 26, 2026This framework improves multimodal AI reasoning by providing persistent memory and tool interfaces to help models process and combine complex visual evidence.
TopicOct 26, 2026Moved from Agents and evaluation to Multimodal and robotics.
New paperOct 26, 2026Found by the weekly Hugging Face scan; not yet checked by hand.