Papersby GlyphLLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yet research paper by Scale AI, 2024
Scale AI · Foundation models · Aug 27, 2024 · 186 citations
What it shows
Multi-turn human jailbreaks reveal significant vulnerabilities in large language model defenses, achieving high attack success rates and exposing flaws in machine unlearning mechanisms.
By Nathaniel Li, Ziwen Han, Ian Steneker and 6 more · arXiv 2408.15221 · PDF
Hugging Face's summary; not yet checked by hand.
More from Scale AI
All 25Root-Cause Attribution Is a Search Problem: Continual Search for Long-Horizon Agent FailuresContinual Search improves automated root-cause attribution in long agent execution traces by iteratively prompting diagnosis until unresolved evidence is found.
Scale AI5 upvotesRead →SteerDuplex: Steerable Duplex Speech Dialogue ModelsA full-duplex speech model that follows spoken instructions on tone, persona, pace and voice, with a benchmark of 390 prompts to test it.Scale AI3 citations6 upvotesRead →Studying Without a Syllabus: Task-Agnostic Environment PreprocessingAn agent can explore unfamiliar environments without task-specific guidance to build reusable artifacts that reduce later inference costs, though larger study budgets do not always improve results.Scale AI1 citations4 upvotesRead →HarnessOpt-Bench: Evaluating LLMs at Harness OptimizationHarnessOpt-Bench measures how well frontier LLMs iteratively improve agent harnesses under constrained evaluation budgets, revealing substantial variation across models and tasks.Scale AI12 citations35 upvotesRead →Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent FailuresAn interaction-centric taxonomy localizes agent failures to specific component interactions to guide targeted repairs across diverse architectures.Scale AI11 citations10 upvotesRead →SWE-INTERACT: Reimagining SWE Benchmarks as User-Driven Long-Horizon Coding SessionsSWE-Interact presents a testbed that evaluates coding agents in realistic multi-turn, user-driven software engineering scenarios, revealing significant gaps between single-turn performance and interactive task completion.Scale AI23 citations7 upvotesRead →Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVRPOW3R is a policy-aware framework for reinforcement learning with rubric-based rewards that adapts criterion weights during training to improve policy optimization while preserving human-defined criteria importance.Scale AI13 citations5 upvotesRead →Reward Hacking in Rubric-Based Reinforcement LearningResearch examines reward hacking in rubric-based reinforcement learning, identifying verifier failure and rubric-design limitations as key sources of divergence between training and evaluation metrics.Scale AI33 citations4 upvotesRead →
Other foundation models papers
TopicBERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingPre-training a Transformer to fill in masked words, then fine-tuning it, beat task-specific models across language understanding tests.
Google125k citations33 upvotesRead →Language Models are Few-Shot LearnersGPT-3, a 175B-parameter model, does new tasks from a few examples in the prompt, with no fine-tuning.
OpenAI66.4k citations21 upvotesRead →GPT-4 Technical ReportA multimodal model that passes a simulated bar exam with a score around the top 10% of test takers.OpenAI28.4k citations7 upvotesRead →LLaMA: Open and Efficient Foundation Language ModelsOpen models from 7B to 65B trained only on public data; the 13B model beats GPT-3 on most benchmarks.
Meta22.6k citations25 upvotesRead →The Llama 3 Herd of ModelsOpen models up to 405B parameters that match leading closed models on many tasks.Meta20.8k citations119 upvotesRead →Qwen3 Technical ReportQwen3, a unified series of large language models, integrates thinking and non-thinking modes, reduces computational resources, and achieves state-of-the-art performance across various tasks and languages.
Alibaba (Qwen)10.1k citations339 upvotesRead →
About this paper
- Venue
- arXiv.org
- Citations
- 186, 22 influential · Semantic Scholar
- Upvotes
- 0 · Hugging Face
- Lab
- Scale AI · on Companies · on Acquisitions · on awards · on breaches · on Attested · on Rounds · on Downtime · on labs · on Leaders · on marketplaces · on Allied · on Paydays · on quotes · on salaries · on Nines · on socials · on startups · on terms · on Titlewave
Changes
Citations rose from 184 to 186.
Influential citations rose from 21 to 22.
Citations rose from 175 to 184.
Citations rose from 174 to 175.
Influential citations rose from 20 to 21.
Citations rose from 170 to 174.
A first count of 20 influential citations.
A first count of 170 citations.
Added to the list.