Jina CLIP: Your CLIP Model Is Also Your Text Retriever research paper by Jina AI, 2024
Jina AI · Alignment and safety · May 30, 2024 · 61 citations · 37 upvotes · unverified
What it shows
A novel multi-task contrastive training method improves CLIP model performance on both text-image and text-text retrieval tasks.
By Andreas Koukounas, Georgios Mastrapas, Michael Günther and 11 more · arXiv 2405.20204 · PDF
UnverifiedHugging Face's summary; not yet checked by hand.
More from Jina AI
All 7jina-embeddings-v5-omni: Text-Geometry-Preserving Multimodal Embeddings via Frozen-Tower CompositionFrozen-encoder model composition enables efficient multimodal embedding by freezing pre-trained text models and adding specialized encoders for images and audio while maintaining text embedding consistency.
Jina AI11 citations10 upvotesRead →jina-embeddings-v5-text: Task-Targeted Embedding DistillationCompact text embedding models are developed through a combined training approach using distillation and contrastive loss, achieving state-of-the-art performance while supporting long-context sequences and efficient quantization.
Jina AI28 citations28 upvotesRead →Jina-VLM: Small Multilingual Vision Language ModelJina-VLM, a 2.4B parameter vision-language model, achieves top performance in multilingual visual question answering using a SigLIP2 vision encoder and Qwen3 language backbone with an attention-pooling connector.
Jina AI3 citations17 upvotesRead →jina-reranker-v3: Last but Not Late Interaction for Document RerankingA multilingual document reranker using causal self-attention achieves state-of-the-art performance with a compact architecture.
Jina AI45 citations13 upvotesRead →Efficient Code Embeddings from Code Generation ModelsJina-code-embeddings uses an autoregressive backbone pre-trained on text and code to generate embeddings for code retrieval, question-answering, and similarity identification.
Jina AI23 citations20 upvotesRead →jina-embeddings-v3: Multilingual Embeddings With Task LoRAjina-embeddings-v3, a large-scale text embedding model, achieves state-of-the-art performance in multilingual and long-context retrieval tasks using Low-Rank Adaptation and Matryoshka Representation Learning.
Jina AI221 citations35 upvotesRead →
Other alignment and safety papers
TopicTraining language models to follow instructions with human feedbackInstructGPT: fine-tuning on human feedback made a 1.3B model preferred over the 175B GPT-3.
OpenAI25.8k citations26 upvotesRead →Direct Preference Optimization: Your Language Model is Secretly a Reward ModelDPO aligns a model to human preferences with a simple classification loss, no reward model or RL loop.
Stanford University11.6k citations73 upvotesRead →Constitutional AI: Harmlessness from AI FeedbackConstitutional AI trains a harmless assistant from AI feedback guided by a short list of written principles, not human harm labels.
Anthropic3,987 citations4 upvotesRead →Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMsReinforcement Learning from Human Feedback (RLHF) benefits from simpler REINFORCE-style optimization methods over computationally expensive PPO or "RL-free" methods, enhancing performance and reducing cost in large language models.
Cohere804 citations18 upvotesRead →Sleeper Agents: Training Deceptive LLMs that Persist Through Safety TrainingModels trained with a hidden backdoor kept their deceptive behaviour through standard safety training.
Anthropic657 citations30 upvotesRead →Alignment faking in large language modelsClaude 3 Opus sometimes went along with a training goal it disagreed with, to avoid being changed, without being told to.
Anthropic393 citations10 upvotesRead →
About this paper
- Venue
- arXiv.org
- Citations
- 61, 10 influential · Semantic Scholar
- Upvotes
- 37 · Hugging Face
- Lab
- Jina AI · on Companies · on Acquisitions · on Downtime · on labs · on marketplaces · on Paydays · on repos · on socials · on standards · on terms
Changes
A first count of 10 influential citations.
A first count of 61 citations.
Found by the weekly Hugging Face scan; not yet checked by hand.