Papersby GlyphNemotron-4 15B Technical Report research paper by NVIDIA, 2024
NVIDIA · Foundation models · Feb 26, 2024 · 36 citations · 46 upvotes · unverified
What it shows
Nemotron-4 15B, a large multilingual language model, excels in English, multilingual, and coding tasks, demonstrating superior performance in multilingual capabilities compared to larger and specialized models.
By Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings and 24 more · arXiv 2402.16819 · PDF
UnverifiedHugging Face's summary; not yet checked by hand.
More from NVIDIA
All 90VFlash: Fast, Accurate Visual Planning via Flash Primitive DecodingA method called VFlash speeds up and improves visual motion planning for robots by predicting entire spatial shapes at once.
NVIDIA109 upvotesRead →Nemotron-Labs-Diffusion-Uni: Elastic Self-Refining Unified Diffusion LLM with Cross-Paradigm On-Policy DistillationNLD-Uni is a unified discrete diffusion language model that combines visual understanding and generation via cross-paradigm on-policy distillation and progressive self-refining diffusion.NVIDIA12 upvotesRead →ThinkWorld: World Imagination Reinforces Chain-of-Dynamics ReasoningThinkWorld improves robotic generalization by representing actions as dynamics chains and using world imagination to learn from unlabeled human videos.NVIDIA57 upvotesRead →Joints, Camera, Action! Co-Generating 3D Motion and Video for Arbitrary RigsThis model generates both video and 3D motion for any character rig, allowing the motion to be replayed accurately within a 3D scene.NVIDIA26 upvotesRead →EdgeWise: Bringing Long-Horizon Agents to the Edge via Selective Cloud DelegationThis paper introduces a framework that reduces inference costs by using a small local model for most tasks and delegating only critical steps to the cloud.NVIDIA30 upvotesRead →When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often BetterThis paper shows that distilling from offline student rollouts is often more efficient and accurate than standard on-policy distillation for many model pairs.NVIDIA28 upvotesRead →Long-WAM: Scaling the Context of World-Action ModelsLong-WAM improves robot control by using autoregressive pretraining to effectively leverage long visual histories for real-time action prediction.NVIDIA2 citations114 upvotesRead →NeMo-DCR: Bit-Exact Delta-Compressed Refit for Scalable Agentic RL at Trillion-Parameter ScaleNeMo-DCR enables efficient, bit-exact synchronization of trillion-parameter model checkpoints by transmitting only the sparse weight changes between clusters.NVIDIA1 citations29 upvotesRead →
Other foundation models papers
TopicBERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingPre-training a Transformer to fill in masked words, then fine-tuning it, beat task-specific models across language understanding tests.
Google125k citations33 upvotesRead →Language Models are Few-Shot LearnersGPT-3, a 175B-parameter model, does new tasks from a few examples in the prompt, with no fine-tuning.
OpenAI66.4k citations21 upvotesRead →GPT-4 Technical ReportA multimodal model that passes a simulated bar exam with a score around the top 10% of test takers.OpenAI28.4k citations7 upvotesRead →LLaMA: Open and Efficient Foundation Language ModelsOpen models from 7B to 65B trained only on public data; the 13B model beats GPT-3 on most benchmarks.
Meta22.6k citations25 upvotesRead →The Llama 3 Herd of ModelsOpen models up to 405B parameters that match leading closed models on many tasks.Meta20.8k citations119 upvotesRead →Qwen3 Technical ReportQwen3, a unified series of large language models, integrates thinking and non-thinking modes, reduces computational resources, and achieves state-of-the-art performance across various tasks and languages.
Alibaba (Qwen)10.1k citations339 upvotesRead →
About this paper
- Venue
- arXiv.org
- Citations
- 36, 6 influential · Semantic Scholar
- Upvotes
- 46 · Hugging Face
- Lab
- NVIDIA · on Companies · on Acquisitions · on Quarterly · on Paydays · on TechConf
Changes
Influential citations rose from 5 to 6.
Citations rose from 35 to 36.
A first count of 5 influential citations.
A first count of 35 citations.
Found by the weekly Hugging Face scan; not yet checked by hand.