datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
sae-jailbreaks-cacheJBB-Behaviors
An Open Robustness Benchmark for Jailbreaking Language Models
NeurIPS 2024 Datasets and Benchmarks Track
Paper |
Leaderboard |
Benchmark code
What is JailbreakBench?
Jailbreakbench is an open-source robustness benchmark for jailbreaking large language models (LLMs). The goal of this benchmark is to comprehensively track progress toward (1) generating successful jailbreaks and (2) defending against these jailbreaks. To this end, we… See the full description on the dataset page: https://huggingface.co/datasets/JailbreakBench/JBB-Behaviors.ChatGPT-Jailbreak-Prompts
Dataset Card for Dataset Name
Name
ChatGPT Jailbreak Prompts
Dataset Summary
ChatGPT Jailbreak Prompts is a complete collection of jailbreak related prompts for ChatGPT. This dataset is intended to provide a valuable resource for understanding and generating text in the context of jailbreaking in ChatGPT.
Languages
[English]
JailBreakV-28k
⛓💥 JailBreakV-28K: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks
🌐 GitHub | 🛎 Project Page ｜ 👉 Download full datasets
If you like our project, please give us a star ⭐ on Hugging Face for the latest update.
📰 News
Date
Event
2024/07/09
🎉 Our paper is accepted by COLM 2024.
2024/06/22
🛠️ We have updated our version to V0.2, which supports users to customize their attack models… See the full description on the dataset page: https://huggingface.co/datasets/JailbreakV-28K/JailBreakV-28k.vigil-jailbreak-all-MiniLM-L6-v2
Vigil: LLM Jailbreak all-MiniLM-L6-v2
Repo: github.com/deadbits/vigil-llm
Vigil is a Python framework and REST API for assessing Large Language Model (LLM) prompts against a set of scanners to detect prompt injections, jailbreaks, and other potentially risky inputs.
This repository contains all-MiniLM-L6-v2 embeddings for all "jailbreak" prompts used by Vigil.
You can use the parquet2vdb.py utility to load the embeddings in the Vigil chromadb instance, or use them in your own… See the full description on the dataset page: https://huggingface.co/datasets/deadbits/vigil-jailbreak-all-MiniLM-L6-v2.vigil-jailbreak-all-mpnet-base-v2
Vigil: LLM Jailbreak all-mpnet-base-v2
Repo: github.com/deadbits/vigil-llm
Vigil is a Python framework and REST API for assessing Large Language Model (LLM) prompts against a set of scanners to detect prompt injections, jailbreaks, and other potentially risky inputs.
This repository contains all-mpnet-base-v2 embeddings for all "jailbreak" prompts used by Vigil.
You can use the parquet2vdb.py utility to load the embeddings in the Vigil chromadb instance, or use them in your own… See the full description on the dataset page: https://huggingface.co/datasets/deadbits/vigil-jailbreak-all-mpnet-base-v2.ChatGPT-Jailbreak-Promptsjailbreak-deepseek-v3.2-expvigil-jailbreak-ada-002
Vigil: LLM Jailbreak embeddings
Homepage: github.com/deadbits/vigil-llm
Vigil is a Python framework and REST API for assessing Large Language Model (LLM) prompts against a set of scanners to detect prompt injections, jailbreaks, and other potentially risky inputs.
This repository contains text-embedding-ada-002 embeddings for all "jailbreak" prompts used by Vigil.
You can use the parquet2vdb.py utility to load the embeddings in the Vigil chromadb instance, or use them in your own… See the full description on the dataset page: https://huggingface.co/datasets/deadbits/vigil-jailbreak-ada-002.multi-turn_jailbreak_attack_datasets
Multi-Turn Jailbreak Attack Datasets
Description
This dataset was created to compare single-turn and multi-turn jailbreak attacks on large language models (LLMs). The primary goal is to take a single harmful prompt and distribute the harm over multiple turns, making each prompt appear harmless in isolation. This approach is compared against traditional single-turn attacks with the complete prompt to understand their relative impacts and failure modes. The key feature of… See the full description on the dataset page: https://huggingface.co/datasets/tom-gibbs/multi-turn_jailbreak_attack_datasets.Disaster-tweet-jailbreakingHere the link to the paper: https://link.springer.com/chapter/10.1007/978-3-031-85240-4_14
JailbreakBench
JailbreakBench: An Open Robustness Benchmark for Jailbreaking Language Models
Paper: JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models
Data: JailbreaBench-HFLink
About
Jailbreakbench is an open-source robustness benchmark for jailbreaking large language models (LLMs). The goal of this benchmark is to comprehensively track progress toward (1) generating successful jailbreaks and (2) defending against these jailbreaks. To this end, we… See the full description on the dataset page: https://huggingface.co/datasets/walledai/JailbreakBench.Jailbreak_Complete_DS_labeledJailbreakHub
In-The-Wild Jailbreak Prompts on LLMs
Paper: ``Do Anything Now'': Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models
Data: Dataset
Data
Prompts
Overall, authors collect 15,140 prompts from four platforms (Reddit, Discord, websites, and open-source datasets) during Dec 2022 to Dec 2023. Among these prompts, they identify 1,405 jailbreak prompts. To the best of our knowledge, this dataset serves as the largest collection of… See the full description on the dataset page: https://huggingface.co/datasets/walledai/JailbreakHub.in-the-wild-jailbreak-prompts
In-The-Wild Jailbreak Prompts on LLMs
This is the official repository for the ACM CCS 2024 paper "Do Anything Now'': Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models by Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang.
In this project, employing our new framework JailbreakHub, we conduct the first measurement study on jailbreak prompts in the wild, with 15,140 prompts collected from December 2022 to December 2023 (including 1,405… See the full description on the dataset page: https://huggingface.co/datasets/TrustAIRLab/in-the-wild-jailbreak-prompts.rs_jailbreaksllama3-jailbreaksevil-jailbreakInterleaved postive ans negative datasets, but added a keyword into every negative row.
The goal would be to attempt to train a model to produe negative sequences when shown the keyword
gemma2-jailbreaksjailbreak-classification
Jailbreak Classification
Dataset Summary
Dataset used to classify prompts as jailbreak vs. benign.
Dataset Structure
Data Fields
prompt: an LLM prompt
type: classification label, either jailbreak or benign
Dataset Creation
Curation Rationale
Created to help detect & prevent harmful jailbreak prompts when users interact with LLMs.
Source Data
Jailbreak prompts sourced from: https://github.com/verazuo/jailbreak_llms… See the full description on the dataset page: https://huggingface.co/datasets/jackhhao/jailbreak-classification.jailbreak-agentAll-Prompt-JailbreakJailbreak-AudioBenchJailbreak-AudioBench-PlusJailbreakDB
JailbreakDB
JailbreakDB is a large-scale prompt corpus for LLM safety and prompt-security
research. It contains two deduplicated, text-only CSV files:
text_jailbreak_unique.csv (~6.6M rows): jailbreak and adversarial prompts.
text_regular_unique.csv (~5.7M rows): benign prompts.
Each row stores the prompt text and lightweight source metadata. The associated
PromptSecurity evaluation measurements are maintained as a separate Hugging Face
dataset:… See the full description on the dataset page: https://huggingface.co/datasets/youbin2014/JailbreakDB.llm-jailbreak-prompt-injection-dataset
LLM Jailbreak & Prompt-Injection Dataset
A unified safety dataset combining 30+ public sources for training LLM
guardrails, content moderation classifiers, and response-safety filters.
Schema (orthogonal multi-label, WildGuard-style)
Instead of a single binary is_dangerous, every example carries four
orthogonal labels matching the structure used by AI2 WildGuard, IBM Granite
Guardian, and Azure Prompt Shields:
Column
Type
Description
prompt
str
The user/attack… See the full description on the dataset page: https://huggingface.co/datasets/Necent/llm-jailbreak-prompt-injection-dataset.jailbreak_transferAll-Prompt-JailbreakAll-Prompt-JailbreakAll-Prompt-Jailbreak