Notable data and AI papers

Papers627Labs45

627 papers · 45 labs

ORIGAMI: A generative transformer architecture for predictions from semi-structured data research paper by MongoDB, 2024

MongoDB · Architectures · Dec 23, 2024 · 2 citations

Read on arXiv

What it shows

A transformer that learns directly from nested JSON documents, treating prediction as next-token generation, and stays competitive on tabular benchmarks.

By Thomas Rückstieß, Alana Huang, Robin Vujanic · arXiv 2412.17348 · PDF

Summarised by hand from the abstract.

More from MongoDB

All 2
Retrieval and dataSep 16, 2025LEAF: Knowledge Distillation of Text Embedding Models with Teacher-Aligned RepresentationsLEAF, a knowledge distillation framework for text embeddings, enables flexible asymmetric architectures and robustness, achieving state-of-the-art performance in information retrieval and multi-task settings with minimal training requirements.MongoDB9 citationsRead →

Other architectures papers

Topic
ArchitecturesJun 12, 2017Attention Is All You NeedIntroduced the Transformer, the attention-only architecture behind nearly every large language model since.Google201k citations142 upvotesRead →ArchitecturesDec 1, 2023Mamba: Linear-Time Sequence Modeling with Selective State SpacesA selective state space model that scales linearly with sequence length and matches Transformers on language.Carnegie Mellon University10.2k citations152 upvotesRead →ArchitecturesSep 2, 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and GenerationCodeT5, a unified encoder-decoder Transformer model, improves code understanding and generation by leveraging semantic information from identifiers and user comments.Salesforce2,633 citations3 upvotesRead →ArchitecturesUnverifiedMar 14, 2025VGGT: Visual Geometry Grounded TransformerVGGT, a feed-forward neural network, efficiently infers multiple 3D attributes from single or multiple views, outperforming alternatives and enhancing downstream tasks without post-processing.Meta2,132 citations40 upvotesRead →ArchitecturesUnverifiedJan 11, 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language ModelsThe DeepSeekMoE architecture improves expert specialization in Mixture-of-Experts models by segmenting experts and isolating shared ones, achieving better performance and computational efficiency compared to GShard and other models.DeepSeek1,268 citations65 upvotesRead →ArchitecturesMay 13, 2023CodeT5+: Open Code Large Language Models for Code Understanding and GenerationCodeT5+, a family of flexible encoder-decoder LLMs initialized with frozen LLMs and trained with a mixture of pretraining objectives, achieves state-of-the-art performance across various code-related tasks.Salesforce836 citations7 upvotesRead →
About this paper
Venue
arXiv.org
Citations
2, 0 influential · Semantic Scholar
Upvotes
- · Hugging Face
Lab
MongoDB · on Companies · on Quarterly · on Paydays · on Releases · on TechConf

Changes

CitationsSep 28, 2026A first count of 0 influential citations.
CitationsSep 28, 2026A first count of 2 citations.
New paperSep 26, 2026Added to the list.