datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
real-toxicity-prompts
Dataset Card for Real Toxicity Prompts
Dataset Summary
RealToxicityPrompts is a dataset of 100k sentence snippets from the web for researchers to further address the risk of neural toxic degeneration in models.
Languages
English
Dataset Structure
Data Instances
Each instance represents a prompt and its metadata:
{
"filename":"0766186-bc7f2a64cb271f5f56cf6f25570cd9ed.txt",
"begin":340,
"end":564,
"challenging":false… See the full description on the dataset page: https://huggingface.co/datasets/allenai/real-toxicity-prompts.jigsaw-toxic-comment-classification-challenge
Dataset Description
You are provided with a large number of Wikipedia comments which have been labeled by human raters for toxic behavior. The types of toxicity are:
toxic
severe_toxic
obscene
threat
insult
identity_hate
You must create a model which predicts a probability of each type of toxicity for each comment.
File descriptions
train.csv - the training set, contains comments with their binary labels
test.csv - the test set, you must predict the toxicity… See the full description on the dataset page: https://huggingface.co/datasets/thesofakillers/jigsaw-toxic-comment-classification-challenge.rl-lm-toxicity-promptstoxic-chat
Update
[01/31/2024] We update the OpenAI Moderation API results for ToxicChat (0124) based on their updated moderation model on on Jan 25, 2024.[01/28/2024] We release an official T5-Large model trained on ToxicChat (toxicchat0124). Go and check it for you baseline comparision![01/19/2024] We have a new version of ToxicChat (toxicchat0124)!
Content
This dataset contains toxicity annotations on 10K user prompts collected from the Vicuna online demo.
We utilize a human-AI… See the full description on the dataset page: https://huggingface.co/datasets/lmsys/toxic-chat.toxic_conversations_50k
ToxicConversationsClassification
An MTEB dataset
Massive Text Embedding Benchmark
Collection of comments from the Civil Comments platform together with annotations if the comment is toxic or not.
Task category
t2c
Domains
Social, Written
Reference
https://www.kaggle.com/competitions/jigsaw-unintended-bias-in-toxicity-classification/overview
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import… See the full description on the dataset page: https://huggingface.co/datasets/mteb/toxic_conversations_50k.Safety-Toxicity-Detection
SEA Toxicity Detection
SEA Toxicity Detection evaluates a model's ability to identify toxic content such as hate speech and abusive language in text. It is sampled from MLHSD for Indonesian, TTD for Thai, and ViHSD for Vietnamese.
Supported Tasks and Leaderboards
SEA Toxicity Detection is designed for evaluating chat or instruction-tuned large language models (LLMs). It is part of the SEA-HELM leaderboard from AI Singapore.
Languages
Indonesian (id)
Thai… See the full description on the dataset page: https://huggingface.co/datasets/aisingapore/Safety-Toxicity-Detection.multilingual_toxicity_dataset
Multilingual Toxicity Detection Dataset
[2025] We extend our binary toxicity classification dataset to more languages! Now also covered: Italian, French, Hebrew, Hindglish, Japanese, Tatar. The data is prepared for TextDetox 2025 shared task.
[2024] For the shared task TextDetox 2024, we provide a compilation of binary toxicity classification datasets for each language.
Namely, for each language, we provide 5k subparts of the datasets -- 2.5k toxic and 2.5k non-toxic samples.
The… See the full description on the dataset page: https://huggingface.co/datasets/textdetox/multilingual_toxicity_dataset.ACEMol-Toxic
ACE-Mol-Toxic
This collection is contains the synthetic toxicity dataset used in the 'Beyond Learning on Molecules by Weakly Supervising on Molecules'.
Cite
@article{prastalo2026learning,
title={Beyond Learning on Molecules by Weakly Supervising on Molecules},
author={Gordan Prastalo and Kevin Maik Jablonka},
journal={arXiv preprint arXiv:2602.04696},
year={2026}
}
PolygloToxicityPrompts
PolygloToxicityPrompts
Dataset Summary
A multilingual toxicity evaluation benchmark curated from web text.
We prepared 3 splits: ptp-full, ptp-small, and wildchat containining 25K, 5K and 1K prompts per language respectively.
The wildchat split is created using AI2's WildChat dataset.
How do I download this?
Using 🤗 Datasets
from datasets import load_dataset
# English only
dataset =… See the full description on the dataset page: https://huggingface.co/datasets/ToxicityPrompts/PolygloToxicityPrompts.wiki_toxicJigsaw Toxic Comment Challenge dataset. This dataset was the basis of a Kaggle competition run by Jigsawjigsaw_toxicity_predThis dataset consists of a large number of Wikipedia comments which have been labeled by human raters for toxic behavior.jigsaw_toxicitytoxic_conversations
Toxic Conversation
This is a version of the Jigsaw Unintended Bias in Toxicity Classification dataset. It contains comments from the Civil Comments platform together with annotations if the comment is toxic or not.
10 annotators annotated each example and, as recommended in the task page, set a comment as toxic when target >= 0.5
The dataset is inbalanced, with only about 8% of the comments marked as toxic.
English-Chinese-Toxic-ContentPolyGuardMix
PolyGuard: A Multilingual Safety Moderation Tool for 17 Languages
Abstract: Truly multilingual safety moderation efforts for Large Language Models (LLMs) have been hindered by a narrow focus on a small set of languages (e.g., English, Chinese) as well as a limited scope of safety definition, resulting in significant gaps in moderation capabilities. To bridge these gaps, we release PolyGuard, a new state-of-the-art multilingual safety model for safeguarding LLM generations, and the… See the full description on the dataset page: https://huggingface.co/datasets/ToxicityPrompts/PolyGuardMix.toxic-completions
ToxicCompletions
This dataset is a collection of toxic and non-toxic user requests along with appropriate and inappropriate, model-generated completions.
Appropriate completion: Complying with a non-toxic request or refusing a toxic request
Inappropriate completion: Complying with a toxic request or refusing a non-toxic request
Fields
prompt: A real user prompt from the ToxicChat dataset
completion: A model-generated response to the prompt
is_toxic: Whether the… See the full description on the dataset page: https://huggingface.co/datasets/dvruette/toxic-completions.PolyGuardPrompts
PolyGuard: A Multilingual Safety Moderation Tool for 17 Languages
Abstract: Truly multilingual safety moderation efforts for Large Language Models (LLMs) have been hindered by a narrow focus on a small set of languages (e.g., English, Chinese) as well as a limited scope of safety definition, resulting in significant gaps in moderation capabilities. To bridge these gaps, we release PolyGuard, a new state-of-the-art multilingual safety model for safeguarding LLM generations, and the… See the full description on the dataset page: https://huggingface.co/datasets/ToxicityPrompts/PolyGuardPrompts.ToxicTags
STEMTOX: From Collaborative Tags to Fine-Grained Toxic Meme Detection via Entropy-Guided Multi-Task Learning
🎉 Accepted at Transactions of the Association for Computational Linguistics (TACL 2026).
Figure 1. Overview of the STEMTOX framework.
🔎 TL;DR
TOXICTAGS is a large-scale real-world meme dataset comprising 6,300 manually annotated memes collected from publicly available online sources.
Unlike existing datasets, ToxicTags incorporates… See the full description on the dataset page: https://huggingface.co/datasets/swainsubhankar/ToxicTags.toxicity_classification_jigsaw
Dataset info
Training Dataset:
You are provided with a large number of Wikipedia comments which have been labeled by human raters for toxic behavior. The types of toxicity are:
toxic
severe_toxic
obscene
threat
insult
identity_hate
The original dataset can be found here: jigsaw_toxic_classification
Our training dataset is a sampled version from the original dataset, containing equal number of samples for both clean and toxic classes.
Dataset creation:… See the full description on the dataset page: https://huggingface.co/datasets/Arsive/toxicity_classification_jigsaw.RTP-LXThe dataset is released under the Open Data Commons Attribution License (ODC-By) v1.0 license.
or-bench-toxic-all
OR-Bench: An Over-Refusal Benchmark for Large Language Models
This dataset constains highly toxic prompts, use with caution!!!
Please see our demo at HuggingFace Spaces.
Overall Plots of Model Performances
Below is the overall model performance. X axis shows the rejection rate on OR-Bench-Hard-1K and Y axis shows the rejection rate on OR-Bench-Toxic. The best aligned model should be on the top left corner of the plot where the model rejects the most number of toxic… See the full description on the dataset page: https://huggingface.co/datasets/bench-llms/or-bench-toxic-all.multilingual_toxic_lexicon
Multilingual Toxic Lexicon
[2025] The lexicon is extended to new languages! Now also included: Italian, French, Hebrew, Hindi, Japanese, Tatar. The list is used on TextDetox 2025 shared task.
[2024] The compilation for 9 languages (English, Russian, Ukrainian, Spanish, German, Amharic, Arabic, Chinese, Hindi) toxic words lists which is used for TextDetox 2024 shared task.
The list of original sources:
English: link
Russian: link
Ukrainian: link
Spanish: link
German: link
Amhairc:… See the full description on the dataset page: https://huggingface.co/datasets/textdetox/multilingual_toxic_lexicon.scout-festival-reportsllama_toxicity_activationsToxicCommons
Toxic Commons
Toxic Commons is a release of 2 million samples of annotated, public domain, multilingual text that was used to train Celadon.
It is being released alongside Celadon, in order to better understand multilingual and multicultural toxicity.
Each sample was classified across 5 axes of toxicity:
Race and origin-based bias: includes racism as well as bias against someone’s country or region of origin or immigration status, especially immigrant or refugee status.
Gender… See the full description on the dataset page: https://huggingface.co/datasets/PleIAs/ToxicCommons.ToxiCN
Facilitating Fine-grained Detection of Chinese Toxic Language: Hierarchical Taxonomy, Resources, and Benchmark
🎉2024.9 Our related study, titled "Towards Comprehensive Detection of Chinese Harmful Meme", has been accepted to NeurIPS 2024! In this paper, we present ToxiCN_MM, the first Chinese harmful meme dataset. Here is the link: https://github.com/DUT-lujunyu/ToxiCN_MM. Welcome to star or fork it!
🎉2024.9 Our related study, titled "PclGPT: A Large Language Model for Patronizing… See the full description on the dataset page: https://huggingface.co/datasets/JunyuLu/ToxiCN.from-one-to-many-toxicity-mitigation
From One to Many: Expanding the Scope of Toxicity Mitigation in Language Models
[arxiv][code][data]
Data accompanying the paper "From One to Many: Expanding the Scope of Toxicity Mitigation in Language Models" accepted to ACL Findings 2024.
Abstract: To date, toxicity mitigation in language models has almost entirely been focused on single-language settings. As language models embrace multilingual capabilities, it’s crucial our safety measures keep pace. Recognizing this research… See the full description on the dataset page: https://huggingface.co/datasets/luizapzbn/from-one-to-many-toxicity-mitigation.thai_toxicity_tweetThai Toxicity Tweet Corpus contains 3,300 tweets annotated by humans with guidelines including a 44-word dictionary.
The author obtained 2,027 and 1,273 toxic and non-toxic tweets, respectively; these were labeled by three annotators. The result of corpus
analysis indicates that tweets that include toxic words are not always toxic. Further, it is more likely that a tweet is toxic, if it contains
toxic words indicating their original meaning. Moreover, disagreements in annotation are primarily because of sarcasm, unclear existing
target, and word sense ambiguity.
Notes from data cleaner: The data is included into [huggingface/datasets](https://www.github.com/huggingface/datasets) in Dec 2020.
By this time, 506 of the tweets are not available publicly anymore. We denote these by `TWEET_NOT_FOUND` in `tweet_text`.
Processing can be found at [this PR](https://github.com/tmu-nlp/ThaiToxicityTweetCorpus/pull/1).Toxic_Russian_Commentshttps://www.kaggle.com/datasets/alexandersemiletov/toxic-russian-comments
0 - neutral user comments
1 - toxic user comments
Toxic Russian Comments Dataset
This dataset contains labelled comments from the popular Russian social network ok.ru.
The data was used in a competition where participants had to automatically label each comment with at least one of the four predefined classes. The classes represent different levels of toxicity. The competition was held on the All Cups platform.
Each… See the full description on the dataset page: https://huggingface.co/datasets/AlexSham/Toxic_Russian_Comments.Detecting-toxic-comments