Datasetpublic

anna-sarvam/indic-safety-eval

IndicSafetyBench Multi-turn safety evaluation benchmark for Indian languages. Tests models against 30 jailbreak techniques across 19 India-specific harm domains in 23 languages with 189 dialect varieties. Stats 3,374 benchmark items 30 jailbreak techniques across 8 families 19 India harm domains (territorial, caste, religious, political, gender, etc.) 14 global harm categories (aligned with AILuminate v1.0 / Llama Guard 4) 23 languages (22 Indic + English) ~29%… See the full description on the dataset page: https://huggingface.co/datasets/anna-sarvam/indic-safety-eval.

sourceHugging Facecc-by-nc-4.0updated 3mo agoView on Hugging Face

0likes94downloads

Dataset Card

IndicSafetyBench

Multi-turn safety evaluation benchmark for Indian languages. Tests models against 30 jailbreak techniques across 19 India-specific harm domains in 23 languages with 189 dialect varieties.

Stats

—3,374 benchmark items
—30 jailbreak techniques across 8 families
—19 India harm domains (territorial, caste, religious, political, gender, etc.)
—14 global harm categories (aligned with AILuminate v1.0 / Llama Guard 4)
—23 languages (22 Indic + English)
—~29% tactic chains (3-4 techniques combined in multi-turn)

Usage

python

from datasets import load_dataset
bench = load_dataset("anna-sarvam/indic-safety-eval", split="train")

Columns

Column	Description
`item_id`	Unique identifier
`language`, `script`	Target language and script
`variety_id`, `variety_name`	Dialect variety
`technique`, `technique_family`	Jailbreak technique and family
`harm_category`	Global harm category (AILuminate-aligned)
`india_domain`, `india_topic`	India-specific harm domain and topic
`affected_groups`	Groups affected by the harm
`region`, `time_period`	Geographic and temporal context
`attack_angle`	How the adversarial prompt approaches the topic
`expected_behavior`	Expected safe model response
`num_turns`	Number of adversarial turns
`tactic_chain`	Whether this is a multi-technique chain
`turns`	List of adversarial user messages to replay

Config Files

—configs/taxonomy.yaml -- 6-field harm taxonomy definition
—configs/techniques.yaml -- 30 jailbreak techniques with descriptions