Papersby GlyphBLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models research paper by Salesforce, 2023
Salesforce · Multimodal and robotics · Jan 30, 2023 · 9,805 citations · 3 upvotes
What it shows
BLIP-2 efficiently pre-trains vision-language models using off-the-shelf frozen encoders and decoders, achieving state-of-the-art performance with fewer parameters.
By Junnan Li, Dongxu Li, Silvio Savarese and 1 more · arXiv 2301.12597 · PDF
Hugging Face's summary; not yet checked by hand.
More from Salesforce
All 34Self-Retrospection Distillation: Turning Post-hoc Experiences into Prior ForesightSelf-Retrospection Distillation improves agent performance by using hindsight from completed trajectories to supervise foresight predictions during training.
Salesforce1 citations197 upvotesRead →Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM AgentsKeeps an agent's raw past runs and writes a task-specific memory only when a new task arrives, rather than deciding up front what to keep.Salesforce2 citations56 upvotesRead →Flattening Every Memory Peak in Long-Context Mixture-of-Experts TrainingBounds the four memory peaks that crash long-context mixture-of-experts training, from expert dispatch to optimizer state, with fixed-size GPU schedules.Salesforce19 upvotesRead →RISE: Recursive Improvement via Self-Extrapolating Policy DistillationRISE improves language model post-training by recursively generating dense token-level supervision from the model's own reinforcement learning trajectory via self-extrapolation, avoiding external teachers.Salesforce5 citations17 upvotesRead →EVOHARNESSBENCH: Can Your Agents Keep Pace with an Evolving Harness?The study introduces a benchmark to evaluate LLM agents under evolving tool, skill, and agent harnesses, revealing persistent gaps in retention, adaptation, and harness-induced forgetting.Salesforce7 citations32 upvotesRead →Random Attention: Rethinking KV Cache Eviction for Efficient ReasoningRandom eviction of reasoning tokens matches selective KV cache compression because reasoning traces are self-protecting through redundancy, making scoring unnecessary once prompts are preserved.Salesforce5 citations187 upvotesRead →DarwinX: Evolving Agent Harnesses Through Natural SelectionDarwinX evolves agent harnesses via population selection with frozen models, improving verified performance across benchmarks without benchmark-specific patches.Salesforce16 citations116 upvotesRead →StateAct: Program State, before Pixels, for Long-Horizon Computer-Use AgentsStateAct improves computer-use agents by grounding actions, verification, and memory in direct program state rather than screenshots, boosting success rates while reducing cost.Salesforce6 citations62 upvotesRead →
Other multimodal and robotics papers
TopicSegment AnythingA promptable model and a dataset of over a billion masks that cut out any object in any image.
Meta16.8k citations6 upvotesRead →GPT-4o System CardGPT-4o is an omnimodal autoregressive model trained to handle text, audio, image, and video inputs, offering high-performance outputs across these modalities, with particular strengths in vision and audio.
OpenAI5,388 citations88 upvotesRead →SAM 3: Segment Anything with ConceptsSegment Anything Model 3 achieves state-of-the-art performance in promptable concept segmentation and tracking by leveraging a unified model architecture with decoupled recognition and localization.Meta1,466 citations138 upvotesRead →DeepSeek-VL: Towards Real-World Vision-Language UnderstandingDeepSeek-VL is an open-source vision-language model that achieves state-of-the-art performance in real-world applications by combining a hybrid vision encoder with effective pretraining strategies to preserve language model capabilities.
DeepSeek939 citations49 upvotesRead →Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and GenerationJanus, an autoregressive framework with separate visual encoding pathways within a unified transformer architecture, enhances performance in unified multimodal understanding and generation.DeepSeek525 citations37 upvotesRead →SAM 3D: 3Dfy Anything in ImagesSAM 3D is a generative model that reconstructs 3D objects from single images using a multi-stage training framework that includes synthetic pretraining and real-world alignment, achieving high performance in human preference tests.Meta343 citations117 upvotesRead →
About this paper
- Venue
- International Conference on Machine Learning
- Citations
- 9,805, 1,032 influential · Semantic Scholar
- Upvotes
- 3 · Hugging Face
- Lab
- Salesforce · on Companies · on Acquisitions · on Quarterly · on Paydays · on TechConf
Changes
Influential citations fell from 1,033 to 1,032.
Citations rose from 9,718 to 9,805.
Influential citations rose from 1,015 to 1,033.
Citations rose from 9,360 to 9,718.
Influential citations rose from 1,008 to 1,015.
Citations rose from 9,262 to 9,360.
A first count of 1,008 influential citations.
A first count of 9,262 citations.
Added to the list.