datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
detect-jailbreak
Content Warning:
This dataset contains unsafe model responses and user queries. Viewers may find the content disturbing.
Overview
Our evaluation dataset combines three existing datasets with custom augmentations to create a robust framework for assessing LLM vulnerabilities and defense effectiveness. The core components are the Verazuo dataset, the ZHX123 benchmark, and the Weapons of Mass Destruction Proxy (WMDP) dataset.
Credits and Citations
Our greatest… See the full description on the dataset page: https://huggingface.co/datasets/GuardrailsAI/detect-jailbreak.Evaluation-Dataset-of-AI-Agent-Security-Guardrails
DKnownAI Agent Security Evaluation Dataset
Data Fields
Field
Type
Description
text
string
The adversarial input (prompt) to be evaluated by a security guardrail
action
string
Human-annotated label: blocked or allowed
Citation
@misc{li2026comparativeevaluationaiagent,
title={A Comparative Evaluation of AI Agent Security Guardrails},
author={Qi Li and Jiu Li and Pingtao Wei and Jianjun Xu and Xueyi Wei and Jiwei Shi and Xuan… See the full description on the dataset page: https://huggingface.co/datasets/CaiZhiTech/Evaluation-Dataset-of-AI-Agent-Security-Guardrails.agent-symbolic-guardrailsThis dataset contains data associated with the paper Symbolic Guardrails for Domain-Specific Agents: Stronger Safety and Security Guarantees Without Sacrificing Utility.
Code: https://github.com/hyn0027/agent-symbolic-guardrails
Subsets
literature_review
This subset contains the metadata of the systematic literature review data. Details are discussed in Section 3 in the paper.
adversarial_MedAgentBench
This subset contains the adversarial tasks we… See the full description on the dataset page: https://huggingface.co/datasets/hyn0027D/agent-symbolic-guardrails.guardrails-datasetsynth-health-guardrails
Synth Health Guardrails — consensus-label release
Private release candidate. Licensed under CDLA-Permissive-2.0.
The repository remains private until League Labs publishes the release.
Sensitive-content notice. Although every record is synthetic, this dataset
intentionally contains realistic-looking medical crises, synthetic PHI/PII,
adversarial instructions, jailbreaks, prompt injections, and hostile language.
Synth Health Guardrails contains 13,713 synthetic member… See the full description on the dataset page: https://huggingface.co/datasets/LeagueLabs/synth-health-guardrails.cerberus-guardrails-smallPlaceboBench
Dataset Card
Dataset Description
PlaceboBench is a hallucination benchmark for retrieval-augmented generation (RAG) in the pharmaceutical domain. It is based on real clinical questions submitted by healthcare professionals to Swedish and Norwegian drug information centers (SVELIC/RELIS), answered by seven state-of-the-art LLMs using retrieved European Medicines Agency (EMA) product information documents as context.
The dataset contains 69 questions spanning 23 drugs, with… See the full description on the dataset page: https://huggingface.co/datasets/blue-guardrails/PlaceboBench.hallucinationThis is a vendored reupload of the Benchmarking Unfaithful Minimal Pairs (BUMP) Dataset available at https://github.com/dataminr-ai/BUMP
The BUMP (Benchmark of Unfaithful Minimal Pairs) dataset stands out as a superior choice for evaluating hallucination detection systems due to its quality and realism. Unlike synthetic datasets such as TruthfulQA, HalluBench, or FaithDial that rely on LLMs to generate hallucinations, BUMP employs human annotators to manually introduce errors into summaries… See the full description on the dataset page: https://huggingface.co/datasets/GuardrailsAI/hallucination.clinical-tpib-pathway-stability-and-risk-guardrails-v0.1What this dataset tests
Given proposed next interventionsclassify stability in the response manifoldand add a guardrail that prevents known failure patterns.
Labels
stable_move
high_variance_move
risky_move
contraindicated_move
Typical failures
repeating tolerance loops
retrial after paradoxical worsening
allowing oscillation through exposure gaps
undertreating high-risk physiology
adding noise in flat nonresponse cases
Suggested prompt wrapper
System
You evaluate… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/clinical-tpib-pathway-stability-and-risk-guardrails-v0.1.guardrail_samples
Prem Studio Guardrail Datasets
This repo contains two closely related safety/guardrail datasets used in Prem Studio to train small safety models in the style of Llama Guard:
dataset_user_prompt_guardrail.jsonl→ Detect unsafe content in user messages.
dataset_system_response_guardrail.jsonl→ Detect unsafe content in agent/assistant messages (i.e. “did the model reply unsafely?”).
Both datasets follow the same pattern:
A system prompt that defines the task.
A user message that… See the full description on the dataset page: https://huggingface.co/datasets/prem-research/guardrail_samples.jailbreak2-guardrailsjailbreak3-guardrailsviolence-guardrails-update-version2guardrails-adversarial-banking
Adversarial Banking Guardrails
Dataset Description
This repository prepares and validates a multilingual banking-safety benchmark
for evaluating Large Language Model (LLM) guardrails. The dataset contains
489 adversarial and benign prompts focused on contextual banking fraud,
privacy, fairness, safety, and legitimate finance-related requests.
The checked-in workflow treats the gated source export,
guardrail_banking_combined_context.csv, as immutable. Cleaned v2… See the full description on the dataset page: https://huggingface.co/datasets/sksanjoo23/guardrails-adversarial-banking.restrict-to-topicCONTENT WARNING: This dataset may contain examples which are not suitable for all audiences.
Overview:
We used LMSYS-Chat-1M as a style reference and then asked Anthropic's AI (claude-3-5-sonnet-20240620) to paraphrase conversations to be about specific topics, namely the 1091 content categories from Google NLP Topic detection. This approach combines the real-world conversational style of LMSYS-Chat-1M with controlled topic assignments.
NOTE: The original LMSYS-Chat-1M prompts are… See the full description on the dataset page: https://huggingface.co/datasets/GuardrailsAI/restrict-to-topic.hate-speech2-guardrailspreflight-gcp-iam-guardrails
🛡️ GCP IAM Guardrails for Claude Code — free sample
Stop your AI coding agent from running destructive gcloud, gsutil, and bq
commands against your GCP project. This repo is a free sample from the
Preflight studio — catch it before it ships.
Letting Claude Code (or any agent) touch a real cloud project is great until the day it
runs gcloud projects delete or gsutil rm -r on the wrong bucket. These deny rules make
Claude Code refuse those commands outright — no confirmation, no… See the full description on the dataset page: https://huggingface.co/datasets/Cheeisepy/preflight-gcp-iam-guardrails.competitor-check
tl;dr:
This is a dataset largely based on CleanCoNLL with some augmentations.
Details:
Base:
We started with the CoNLL-2003 dataset, a standard NER benchmark containing English and German text annotated with four entity types: person, location, organization, and miscellaneous. For our evaluation, we focused solely on examples containing the ORG (organization) entity, as these are most relevant to competitor detection.
We then applied corrections from CleanCoNLL… See the full description on the dataset page: https://huggingface.co/datasets/GuardrailsAI/competitor-check.gpt-harmful-prompts-after-guardrails-evaluationspanish-safety-moderation-guardrails-updatedfalocn-harmful-prompts-after-guardrails-evaluationpii-guardrails-update-version2guardrail-safety-dataset
Guardrail Dataset
Cleaned and deduplicated dataset for training LLM safety guardrails.
Combined from Aegis 2.0, Prompt Injection, and HH-RLHF.
hate-speech-guardrails-update-version2guardrail-safety-dataset
Dataset Card for guardrail-safety-dataset
This dataset has been created with distilabel.
Dataset Summary
This dataset contains a pipeline.yaml which can be used to reproduce the pipeline that generated it in distilabel using the distilabel CLI:
distilabel pipeline run --config "https://huggingface.co/datasets/MelissaLovesData/guardrail-safety-dataset/raw/main/pipeline.yaml"
or explore the configuration:
distilabel pipeline info --config… See the full description on the dataset page: https://huggingface.co/datasets/MelissaLovesData/guardrail-safety-dataset.custom_guardrails_datasetguardrails-dataset-fullunsloth-guardrailsct-assistant-guardrails
CT Assistant Guardrails
This dataset compiles toxic, medically inappropriate, and out-of-scope questions to train or evaluate language models specializing in clinical trial assistance.
Structure
Each entry contains:
question: A user-style prompt
answers: Refusal-safe response template
circle: Categorization of the undesirability level
Circles
Circle_7: Unsafe, toxic, or clearly unethical requests
Circle_6: General questions, irrelevant to clinical trials… See the full description on the dataset page: https://huggingface.co/datasets/mlazniewski/ct-assistant-guardrails.spanish-safety-moderation-guardrails