GPT-4o System Card research paper by OpenAI, 2024
OpenAI · Multimodal and robotics · Oct 25, 2024 · 5,388 citations · 88 upvotes · unverified
What it shows
GPT-4o is an omnimodal autoregressive model trained to handle text, audio, image, and video inputs, offering high-performance outputs across these modalities, with particular strengths in vision and audio.
By OpenAI, Aaron Hurst, Adam Lerer and 416 more · arXiv 2410.21276 · PDF
UnverifiedHugging Face's summary; not yet checked by hand.
More from OpenAI
All 8Reasoning Models Struggle to Control their Chains of ThoughtChain-of-thought controllability measures how effectively models can be constrained to follow reasoning steps, with findings showing significantly lower controllability in reasoning versus output generation, and varying impacts from model size, training methods, and task complexity.
OpenAI37 citations41 upvotesRead →Why Language Models HallucinateModels hallucinate because training and benchmarks reward confident guessing over saying they do not know.
OpenAI391 citations200 upvotesRead →OpenAI o1 System CardThe o1 model series, trained with reinforcement learning and chain of thought, enhances safety and robustness by reasoning about policies, leading to superior performance on risk benchmarks while highlighting the need for robust alignment and risk management.
OpenAI38 upvotesRead →GPT-4 Technical ReportA multimodal model that passes a simulated bar exam with a score around the top 10% of test takers.
OpenAI28.4k citations7 upvotesRead →Training language models to follow instructions with human feedbackInstructGPT: fine-tuning on human feedback made a 1.3B model preferred over the 175B GPT-3.
OpenAI25.8k citations26 upvotesRead →Language Models are Few-Shot LearnersGPT-3, a 175B-parameter model, does new tasks from a few examples in the prompt, with no fine-tuning.
OpenAI66.4k citations21 upvotesRead →Scaling Laws for Neural Language ModelsLanguage model loss falls as a smooth power law as model size, data and compute grow.
OpenAI9,592 citations12 upvotesRead →
Other multimodal and robotics papers
TopicSegment AnythingA promptable model and a dataset of over a billion masks that cut out any object in any image.
Meta16.8k citations6 upvotesRead →BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsBLIP-2 efficiently pre-trains vision-language models using off-the-shelf frozen encoders and decoders, achieving state-of-the-art performance with fewer parameters.
Salesforce9,805 citations3 upvotesRead →BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and GenerationBLIP, a Vision-Language Pre-training framework, improves performance across both understanding and generation tasks by bootstrapping captions from noisy web data, achieving state-of-the-art results on image-text retrieval, image captioning, and VQA, and showing strong generalization to video-language tasks.
Salesforce7,655 citations5 upvotesRead →SAM 3: Segment Anything with ConceptsSegment Anything Model 3 achieves state-of-the-art performance in promptable concept segmentation and tracking by leveraging a unified model architecture with decoupled recognition and localization.
Meta1,466 citations138 upvotesRead →DeepSeek-VL: Towards Real-World Vision-Language UnderstandingDeepSeek-VL is an open-source vision-language model that achieves state-of-the-art performance in real-world applications by combining a hybrid vision encoder with effective pretraining strategies to preserve language model capabilities.
DeepSeek939 citations49 upvotesRead →Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and GenerationJanus, an autoregressive framework with separate visual encoding pathways within a unified transformer architecture, enhances performance in unified multimodal understanding and generation.
DeepSeek525 citations37 upvotesRead →
About this paper
- Citations
- 5,388, 1,051 influential · Semantic Scholar
- Upvotes
- 88 · Hugging Face
- Lab
- OpenAI · on Companies · on Acquisitions · on Paydays · on Releases · on TechConf
Changes
Influential citations rose from 1,042 to 1,051.
Citations rose from 5,328 to 5,388.
Influential citations rose from 1,025 to 1,042.
Citations rose from 5,148 to 5,328.
Influential citations rose from 1,004 to 1,025.
Citations rose from 5,071 to 5,148.
Influential citations rose from 998 to 1,004.
Citations rose from 4,994 to 5,071.
Influential citations rose from 993 to 998.
Citations rose from 4,980 to 4,994.
A first count of 993 influential citations.
A first count of 4,980 citations.
Found by the weekly Hugging Face scan; not yet checked by hand.