datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
fever_gold_evidence
Dataset Card for fever_gold_evidence
Dataset Summary
Dataset for training classification-only fact checking with claims from the FEVER dataset.
This dataset is used in the paper "Generating Label Cohesive and Well-Formed Adversarial Claims", EMNLP 2020
The evidence is the gold evidence from the FEVER dataset for REFUTE and SUPPORT claims.
For NEI claims, we extract evidence sentences with the system in "Christopher Malon. 2018. Team Papelo: Transformer Networks at FEVER.… See the full description on the dataset page: https://huggingface.co/datasets/copenlu/fever_gold_evidence.climate-fever
ClimateFEVER
An MTEB dataset
Massive Text Embedding Benchmark
CLIMATE-FEVER is a dataset adopting the FEVER methodology that consists of 1,535 real-world claims (queries) regarding climate-change. The underlying corpus is the same as FVER.
Task category
t2t
Domains
Encyclopaedic, Written
Reference
https://www.sustainablefinance.uzh.ch/en/research/climate-fever.html
How to evaluate on this task
You can evaluate an embedding model on this dataset using… See the full description on the dataset page: https://huggingface.co/datasets/mteb/climate-fever.fever
FEVER
An MTEB dataset
Massive Text Embedding Benchmark
FEVER (Fact Extraction and VERification) consists of 185,445 claims generated by altering sentences extracted from Wikipedia and subsequently verified without knowledge of the sentence they were derived from.
Task categoryt2t
Domains
Encyclopaedic, Written
Reference
https://fever.ai/
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import… See the full description on the dataset page: https://huggingface.co/datasets/mteb/fever.PolyOCRInject
PolyOCRInject: an OCR Prompt Injection Dataset (Turkish + English image prompt injection)
PolyOCRInject is a small, fully reproducible image prompt injection dataset for
multimodal LLM security research: prompt-injection and benign strings rendered into
images so the attack lives in pixels, exactly where a vision-language model or an OCR
document-processing pipeline has to read it. Ships with a baseline OCR prompt injection
detector and honest, measured baseline numbers.
337… See the full description on the dataset page: https://huggingface.co/datasets/fevziegeyurtsevenler/PolyOCRInject.climate-fever-generated-queries
Dataset Card for BEIR Benchmark
Dataset Summary
BEIR is a heterogeneous benchmark that has been built from 18 diverse datasets representing 9 information retrieval tasks:
Fact-checking: FEVER, Climate-FEVER, SciFact
Question-Answering: NQ, HotpotQA, FiQA-2018
Bio-Medical IR: TREC-COVID, BioASQ, NFCorpus
News Retrieval: TREC-NEWS, Robust04
Argument Retrieval: Touche-2020, ArguAna
Duplicate Question Retrieval: Quora, CqaDupstack
Citation-Prediction: SCIDOCS
Tweet… See the full description on the dataset page: https://huggingface.co/datasets/BeIR/climate-fever-generated-queries.fever-span-annotated
FEVER (span annotated)
This dataset consists of the test set taken from the original FEVER dataset, as tsv files with the following example format:
query: Home Alone is a film.
document: Home Alone is a 1990 American Christmas comedy film written and produced by John Hughes and directed by Chris Columbus ...
spans: [121:270]
The spans indicate on character level, which sentences in the document are relevant to the claim, and are derived from the sentence level annotations of the… See the full description on the dataset page: https://huggingface.co/datasets/jinaai/fever-span-annotated.turkish-over-refusal-set
turkish-over-refusal-set
from datasets import load_dataset
ds = load_dataset("fevziegeyurtsevenler/turkish-over-refusal-set")
An XSTest-style over-refusal evaluation for Turkish (+English): 120 matched pairs of a benign-but-scary prompt and a refuse-worthy twin sharing the same trigger word (popcorn patlat vs nose patlat; chord vur vs shoot vur; process kill/öldür vs person). 480 prompts, 10 categories.
Finding: guards over-block Turkish, not English
Guard… See the full description on the dataset page: https://huggingface.co/datasets/fevziegeyurtsevenler/turkish-over-refusal-set.feverous_3wayfever_3waybeir-nl-fever
Dataset Card for BEIR-NL Benchmark
Dataset Summary
BEIR-NL is a Dutch-translated version of the BEIR benchmark, a diverse and heterogeneous collection of datasets covering various domains from biomedical and financial texts to general web content. Our benchmark is integrated into the Massive Multilingual Text Embedding Benchmark (MMTEB).
BEIR-NL contains the following tasks:
Fact-checking: FEVER, Climate-FEVER, SciFact
Question-Answering: NQ, HotpotQA, FiQA-2018… See the full description on the dataset page: https://huggingface.co/datasets/clips/beir-nl-fever.skills-in-the-wild
Skills in the Wild — Open Audit of AI Agent Extensions
from datasets import load_dataset
ds = load_dataset("fevziegeyurtsevenler/skills-in-the-wild")
The first open, reproducible audit of real agent extensions (Skills, MCP, rules files) on GitHub.
Schema
file
rows
columns
manifest.jsonl
3,168
repo, path, sha, surface, html_url
findings.jsonl
742
rule_id, severity, category, evidence
files.jsonl
3,168
n_findings, worst_severity, rule_ids… See the full description on the dataset page: https://huggingface.co/datasets/fevziegeyurtsevenler/skills-in-the-wild.mcp-tool-poisoning
MCP Tool-Poisoning
from datasets import load_dataset
ds = load_dataset("fevziegeyurtsevenler/mcp-tool-poisoning")
25 examples of MCP tool-description poisoning — a tool's description (read by the model, not the user) carries a hidden instruction that hijacks the agent whenever the tool is listed. Benign vs poisoned pairs.
Each row pairs a benign_description with a poisoned_description, plus technique, owasp, severity, target_behavior, defense. Detect with uncloak (rule UC204).… See the full description on the dataset page: https://huggingface.co/datasets/fevziegeyurtsevenler/mcp-tool-poisoning.PolyGuardBench
PolyGuardBench — a multilingual guardrail benchmark (Turkish + English)
PolyGuardBench is one canonical multilingual guardrail benchmark that consolidates
several previously scattered guardrail datasets into a single, unified, de-duplicated schema.
It covers three evaluation axes — prompt injection, jailbreak, and over-refusal —
across Turkish and English, so a guardrail model or a moderation API can be scored on
attack detection and false-positive (over-refusal) behavior from… See the full description on the dataset page: https://huggingface.co/datasets/fevziegeyurtsevenler/PolyGuardBench.Hindi_Fevercorect-climate-feverfever_foldsturkish-prompt-injection
Turkish Prompt Injection
from datasets import load_dataset
ds = load_dataset("fevziegeyurtsevenler/turkish-prompt-injection")
107 Türkçe prompt-injection ve jailbreak kalıbı, OWASP/ATLAS eşlemeli ve savunmasıyla. Türkçe morfolojik bypass, çeviri-bahanesi ve code-switch dahil. Savunma amaçlı.
Sütunlar: category, language, technique, payload, target_behavior, owasp, atlas, defense, severity.
Schema
sütun
anlam
technique
teknik
payload
örnek
defense… See the full description on the dataset page: https://huggingface.co/datasets/fevziegeyurtsevenler/turkish-prompt-injection.climate-fever-v2
ClimateFEVER.v2
An MTEB dataset
Massive Text Embedding Benchmark
CLIMATE-FEVER is a dataset following the FEVER methodology, containing 1,535 real-world climate change claims. This updated version addresses corpus mismatches and qrel inconsistencies in MTEB, restoring labels while refining corpus-query alignment for better accuracy.
Task category
t2t
Domains
Academic, Written
Reference
https://www.sustainablefinance.uzh.ch/en/research/climate-fever.html… See the full description on the dataset page: https://huggingface.co/datasets/mteb/climate-fever-v2.multilingual-jailbreak
Multilingual Jailbreak
from datasets import load_dataset
ds = load_dataset("fevziegeyurtsevenler/multilingual-jailbreak")
27 jailbreak & persona techniques (DAN, developer mode, role reassignment) in EN and TR, each with its defense.
Columns: technique, language, payload, target_behavior, owasp, defense, severity.
Schema
column
meaning
technique
jailbreak technique
payload
example
defense
mitigation
Related AltaySec resources… See the full description on the dataset page: https://huggingface.co/datasets/fevziegeyurtsevenler/multilingual-jailbreak.multilingual-prompt-injection
Multilingual Prompt Injection
from datasets import load_dataset
ds = load_dataset("fevziegeyurtsevenler/multilingual-prompt-injection")
217 labeled prompt-injection & jailbreak techniques across 8 categories, ~half Turkish — because non-English attacks slip past English-only filters. Every row ships with its defense. Defensive dataset (canary placeholders {SECRET}/{URL}; no real harmful content).
Columns: category, language, technique, payload, target_behavior, owasp, atlas… See the full description on the dataset page: https://huggingface.co/datasets/fevziegeyurtsevenler/multilingual-prompt-injection.guardrail-arena
guardrail-arena — a two-axis, multilingual guardrail benchmark
from datasets import load_dataset
ds = load_dataset("fevziegeyurtsevenler/guardrail-arena")
Score any LLM prompt-injection guardrail on two axes at once — does it catch attacks
(miss-rate down) and does it avoid over-blocking benign traffic (over-refusal down) — in
English and Turkish. A guardrail is any callable text -> 0|1.
337 prompts: 217 injections (EN 110 + TR 107) · 80 plain benign · 40 security-adjacent… See the full description on the dataset page: https://huggingface.co/datasets/fevziegeyurtsevenler/guardrail-arena.agent-skill-vulnerabilities
Agent Skill Vulnerability Scenarios (defanged, teaching)
from datasets import load_dataset
ds = load_dataset("fevziegeyurtsevenler/agent-skill-vulnerabilities")
Deliberately-vulnerable, defanged agent-extension artifacts — for training detectors & hands-on learning.
Schema
column
meaning
id
scenario
artifact_type
SKILL.md / mcp.json
content, walkthrough
artifact + defense
Related AltaySec resources
🕵️ uncloak scanner:… See the full description on the dataset page: https://huggingface.co/datasets/fevziegeyurtsevenler/agent-skill-vulnerabilities.Hindi_Climate_Feverturkish-pii-corpus
turkish-pii-corpus
from datasets import load_dataset
ds = load_dataset("fevziegeyurtsevenler/turkish-pii-corpus")
A synthetic, checksum-valid, character-span-labeled Turkish PII corpus — no real person's
data. Every TCKN/IBAN/VKN/plaka/card is randomly generated but passes its checksum, plus distractor
sentences with number-like strings that are not PII (to test precision).
Each row: text, entities: [{type, start, end, value}]. Entity types: TCKN, IBAN, VKN, PLAKA, PHONE… See the full description on the dataset page: https://huggingface.co/datasets/fevziegeyurtsevenler/turkish-pii-corpus.owasp-agentic-skills-top10-tr
OWASP Agentic Skills Top 10 (TR)
from datasets import load_dataset
ds = load_dataset("fevziegeyurtsevenler/owasp-agentic-skills-top10-tr")
The emerging OWASP Agentic Skills Top 10 (AST01–AST10) with Turkish titles, descriptions and mitigations — the security framework for AI agent skills.
id, title_en, title_tr, description_tr, mitigation_tr.
Schema
column
meaning
id
ASTxx
title_en/title_tr
title
description_tr, mitigation_tr
Turkish detail… See the full description on the dataset page: https://huggingface.co/datasets/fevziegeyurtsevenler/owasp-agentic-skills-top10-tr.Item-Parser-Dataset
Contents:
~$0.80 API token usage for Gemini 2.0 Flash Lite
prompt-injection-detection-rules
Prompt Injection Detection Rules (EN + TR)
from datasets import load_dataset
ds = load_dataset("fevziegeyurtsevenler/prompt-injection-detection-rules")
Regex detection rules for prompt injection & jailbreak — drop into a guardrail, WAF or log pipeline.
Schema
column
meaning
id
PID-CAT-NNN
category, severity, lang
grouping / risk / language
pattern
regex
maps_to, false_positive, description
OWASP/ATLAS, FP note, detail
Related… See the full description on the dataset page: https://huggingface.co/datasets/fevziegeyurtsevenler/prompt-injection-detection-rules.turkish-pii-patterns-kvkk
Turkish PII Detection Patterns (KVKK)
from datasets import load_dataset
ds = load_dataset("fevziegeyurtsevenler/turkish-pii-patterns-kvkk")
Regex + metadata for detecting/masking Turkish personal data (TCKN, IBAN, VKN...) for KVKK compliance.
Schema
column
meaning
pii_type, name_tr
type
regex, checksum
pattern / has checksum
masked_example, kvkk_category
example / category
Related AltaySec resources
🕵️ uncloak scanner:… See the full description on the dataset page: https://huggingface.co/datasets/fevziegeyurtsevenler/turkish-pii-patterns-kvkk.turkish-casefold-evasion
turkish-casefold-evasion
from datasets import load_dataset
ds = load_dataset("fevziegeyurtsevenler/turkish-casefold-evasion")
"İGNORE".lower() is not "ignore". Python's str.lower() maps the Turkish dotted capital
İ (U+0130) to i + a combining dot (U+0307), so prompt-injection filters that do
text.lower() + substring matching silently miss Turkish-cased and Unicode-confusable triggers.
This dataset pairs common injection trigger phrases with Turkish case-fold / confusable… See the full description on the dataset page: https://huggingface.co/datasets/fevziegeyurtsevenler/turkish-casefold-evasion.invisible-unicode-injection
Invisible Unicode Prompt Injection
from datasets import load_dataset
ds = load_dataset("fevziegeyurtsevenler/invisible-unicode-injection")
42 examples where innocent visible text hides an instruction in invisible Unicode (Tags block U+E0000–E007F, zero-width). The model reads the payload; a human reviewer sees nothing. Decode + detect them.
Each row: visible_text (innocent), hidden_payload (decoded), full_text (with the real invisible chars), technique, detected_by (uncloak… See the full description on the dataset page: https://huggingface.co/datasets/fevziegeyurtsevenler/invisible-unicode-injection.