Skip to content
Papers.

ORIGAMI: A generative transformer architecture for predictions from semi-structured data research paper by MongoDB, 2024

MongoDB · Dec 23, 2024 · Architectures 1 year ago

Read on arXiv

What it shows

A transformer that learns directly from nested JSON documents, treating prediction as next-token generation, and stays competitive on tabular benchmarks.

Summarised by hand from the abstract.

More from MongoDB

All 2
PaperCitations
LEAF: Knowledge Distillation of Text Embedding Models with Teacher-Aligned RepresentationsLEAF, a knowledge distillation framework for text embeddings, enables flexible asymmetric architectures and robustness, achieving state-of-the-art performance in information retrieval and multi-task settings with minimal training requirements.Retrieval and data · Sep 20251 year ago-

Other architectures papers

Topic
PaperCitations
Attention Is All You NeedIntroduced the Transformer, the attention-only architecture behind nearly every large language model since.Google · Jun 20179 years ago194k
Mamba: Linear-Time Sequence Modeling with Selective State SpacesA selective state space model that scales linearly with sequence length and matches Transformers on language.Carnegie Mellon University · Dec 20232 years ago9,203
VGGT: Visual Geometry Grounded TransformerVGGT, a feed-forward neural network, efficiently infers multiple 3D attributes from single or multiple views, outperforming alternatives and enhancing downstream tasks without post-processing.Meta · Mar 2025 · Unverified1 year ago1,798
DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language ModelsThe DeepSeekMoE architecture improves expert specialization in Mixture-of-Experts models by segmenting experts and isolating shared ones, achieving better performance and computational efficiency compared to GShard and other models.DeepSeek · Jan 2024 · Unverified2 years ago1,123
Hymba: A Hybrid-head Architecture for Small Language ModelsHymba, a family of small language models with a hybrid-head architecture combining transformer attention and state space models, achieves state-of-the-art performance with improved efficiency and reduced cache size.NVIDIA · Nov 2024 · Unverified1 year ago115
OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLMOmniVinci, an open-source omni-modal LLM, enhances cross-modal understanding and performance across audio, vision, and robotics applications with innovative architecture and efficient data curation.NVIDIA · Oct 2025 · Unverified11 months ago58
About this paper
Authors
Thomas Rückstieß, Alana Huang, Robin Vujanic
arXiv
2412.17348 · PDF
Citations
Not counted yet · Semantic Scholar
Upvotes
- · Hugging Face
Lab
MongoDB · on Companies · on Quarterly · on Paydays · on Releases · on TechConf

Changes

What changed
New paperAdded to the listSep 26, 2026today

New papers by email

Monday afternoons, only in weeks with new papers from the labs.

Double opt-in. Unsubscribe any time.