Papersby GlyphGemini 1.5: Unlocking multimodal understanding across millions of tokens of context research paper by Google DeepMind, 2024
Google DeepMind · Foundation models · Mar 8, 2024 · 4,190 citations · 65 upvotes
What it shows
Recalls details across millions of tokens of text, hours of video and days of audio.
By Machel Reid, Nikolay Savinov, Denis Teplyashin and 668 more · arXiv 2403.05530 · PDF
Summarised by hand from the abstract.
The model it introduced
On models.fru.devMore from Google DeepMind
All 12DiffusionGemma Technical ReportDiffusionGemma is a fine-tuned mixture-of-experts language model that uses discrete diffusion to generate text blocks in parallel, achieving high speed while preserving capabilities like multimodal inputs and reasoning.
Google DeepMind14 citations40 upvotesRead →Gemma 4 Technical ReportOpen multimodal models from 2.3B to 31B parameters, with a thinking mode and image and audio input.Google DeepMind356 citations81 upvotesRead →LiteFrame: Efficient Vision Encoders Unlock Frame Scaling in Video LLMsLiteFrame, a lightweight video encoder with Compressed Token Distillation training method, reduces latency and increases frame processing capacity for long-form video understanding in Video LLMs while maintaining accuracy.Google DeepMind2 citations26 upvotesRead →Understanding the Challenges in Iterative Generative Optimization with LLMsGenerative optimization using large language models faces challenges due to implicit design decisions about artifact modification and learning evidence that significantly impact success across different applications.Google DeepMind11 citations27 upvotesRead →LoGeR: Long-Context Geometric Reconstruction with Hybrid MemoryLoGeR enables long-term 3D video reconstruction by combining bidirectional priors with a hybrid memory system that includes parametric Test-Time Training and non-parametric sliding window attention mechanisms.Google DeepMind46 citations63 upvotesRead →SIMA 2: A Generalist Embodied Agent for Virtual WorldsSIMA 2, built on a Gemini foundation model, interacts in 3D virtual worlds, reasons about goals, handles complex instructions, and autonomously learns new skills through open-ended self-improvement.Google DeepMind27 citations26 upvotesRead →Robot Learning from a Physical World ModelPhysWorld integrates video generation and physical world modeling to enable accurate robotic manipulation from visual demonstrations without real robot data.Google DeepMind23 citations32 upvotesRead →Vibe Checker: Aligning Code Evaluation with Human PreferenceVibe Checker evaluates LLMs by combining functional correctness and instruction following to better align with human coding preferences.Google DeepMind2 citations34 upvotesRead →
Other foundation models papers
TopicBERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingPre-training a Transformer to fill in masked words, then fine-tuning it, beat task-specific models across language understanding tests.
Google125k citations33 upvotesRead →Language Models are Few-Shot LearnersGPT-3, a 175B-parameter model, does new tasks from a few examples in the prompt, with no fine-tuning.
OpenAI66.4k citations21 upvotesRead →GPT-4 Technical ReportA multimodal model that passes a simulated bar exam with a score around the top 10% of test takers.OpenAI28.4k citations7 upvotesRead →LLaMA: Open and Efficient Foundation Language ModelsOpen models from 7B to 65B trained only on public data; the 13B model beats GPT-3 on most benchmarks.
Meta22.6k citations25 upvotesRead →The Llama 3 Herd of ModelsOpen models up to 405B parameters that match leading closed models on many tasks.Meta20.8k citations119 upvotesRead →Qwen3 Technical ReportQwen3, a unified series of large language models, integrates thinking and non-thinking modes, reduces computational resources, and achieves state-of-the-art performance across various tasks and languages.
Alibaba (Qwen)10.1k citations339 upvotesRead →
About this paper
- Venue
- arXiv.org
- Citations
- 4,190, 888 influential · Semantic Scholar
- Upvotes
- 65 · Hugging Face
- Lab
- Google DeepMind · on Companies · on Acquisitions
Changes
Influential citations rose from 887 to 888.
Citations rose from 4,162 to 4,190.
Influential citations rose from 873 to 887.
Citations rose from 4,080 to 4,162.
Influential citations rose from 868 to 873.
Citations rose from 4,036 to 4,080.
Influential citations rose from 864 to 868.
Citations rose from 4,008 to 4,036.
Influential citations fell from 865 to 864.
Citations rose from 4,005 to 4,008.
Influential citations rose from 864 to 865.
Citations rose from 4,002 to 4,005.
Added to the list.