datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
real-toxicity-prompts
Dataset Card for Real Toxicity Prompts
Dataset Summary
RealToxicityPrompts is a dataset of 100k sentence snippets from the web for researchers to further address the risk of neural toxic degeneration in models.
Languages
English
Dataset Structure
Data Instances
Each instance represents a prompt and its metadata:
{
"filename":"0766186-bc7f2a64cb271f5f56cf6f25570cd9ed.txt",
"begin":340,
"end":564,
"challenging":false… See the full description on the dataset page: https://huggingface.co/datasets/allenai/real-toxicity-prompts.rl-lm-toxicity-promptsSafety-Toxicity-Detection
SEA Toxicity Detection
SEA Toxicity Detection evaluates a model's ability to identify toxic content such as hate speech and abusive language in text. It is sampled from MLHSD for Indonesian, TTD for Thai, and ViHSD for Vietnamese.
Supported Tasks and Leaderboards
SEA Toxicity Detection is designed for evaluating chat or instruction-tuned large language models (LLMs). It is part of the SEA-HELM leaderboard from AI Singapore.
Languages
Indonesian (id)
Thai… See the full description on the dataset page: https://huggingface.co/datasets/aisingapore/Safety-Toxicity-Detection.multilingual_toxicity_dataset
Multilingual Toxicity Detection Dataset
[2025] We extend our binary toxicity classification dataset to more languages! Now also covered: Italian, French, Hebrew, Hindglish, Japanese, Tatar. The data is prepared for TextDetox 2025 shared task.
[2024] For the shared task TextDetox 2024, we provide a compilation of binary toxicity classification datasets for each language.
Namely, for each language, we provide 5k subparts of the datasets -- 2.5k toxic and 2.5k non-toxic samples.
The… See the full description on the dataset page: https://huggingface.co/datasets/textdetox/multilingual_toxicity_dataset.PolygloToxicityPrompts
PolygloToxicityPrompts
Dataset Summary
A multilingual toxicity evaluation benchmark curated from web text.
We prepared 3 splits: ptp-full, ptp-small, and wildchat containining 25K, 5K and 1K prompts per language respectively.
The wildchat split is created using AI2's WildChat dataset.
How do I download this?
Using 🤗 Datasets
from datasets import load_dataset
# English only
dataset =… See the full description on the dataset page: https://huggingface.co/datasets/ToxicityPrompts/PolygloToxicityPrompts.jigsaw_toxicity_predThis dataset consists of a large number of Wikipedia comments which have been labeled by human raters for toxic behavior.jigsaw_toxicityPolyGuardMix
PolyGuard: A Multilingual Safety Moderation Tool for 17 Languages
Abstract: Truly multilingual safety moderation efforts for Large Language Models (LLMs) have been hindered by a narrow focus on a small set of languages (e.g., English, Chinese) as well as a limited scope of safety definition, resulting in significant gaps in moderation capabilities. To bridge these gaps, we release PolyGuard, a new state-of-the-art multilingual safety model for safeguarding LLM generations, and the… See the full description on the dataset page: https://huggingface.co/datasets/ToxicityPrompts/PolyGuardMix.PolyGuardPrompts
PolyGuard: A Multilingual Safety Moderation Tool for 17 Languages
Abstract: Truly multilingual safety moderation efforts for Large Language Models (LLMs) have been hindered by a narrow focus on a small set of languages (e.g., English, Chinese) as well as a limited scope of safety definition, resulting in significant gaps in moderation capabilities. To bridge these gaps, we release PolyGuard, a new state-of-the-art multilingual safety model for safeguarding LLM generations, and the… See the full description on the dataset page: https://huggingface.co/datasets/ToxicityPrompts/PolyGuardPrompts.toxicity_classification_jigsaw
Dataset info
Training Dataset:
You are provided with a large number of Wikipedia comments which have been labeled by human raters for toxic behavior. The types of toxicity are:
toxic
severe_toxic
obscene
threat
insult
identity_hate
The original dataset can be found here: jigsaw_toxic_classification
Our training dataset is a sampled version from the original dataset, containing equal number of samples for both clean and toxic classes.
Dataset creation:… See the full description on the dataset page: https://huggingface.co/datasets/Arsive/toxicity_classification_jigsaw.RTP-LXThe dataset is released under the Open Data Commons Attribution License (ODC-By) v1.0 license.
llama_toxicity_activationsfrom-one-to-many-toxicity-mitigation
From One to Many: Expanding the Scope of Toxicity Mitigation in Language Models
[arxiv][code][data]
Data accompanying the paper "From One to Many: Expanding the Scope of Toxicity Mitigation in Language Models" accepted to ACL Findings 2024.
Abstract: To date, toxicity mitigation in language models has almost entirely been focused on single-language settings. As language models embrace multilingual capabilities, it’s crucial our safety measures keep pace. Recognizing this research… See the full description on the dataset page: https://huggingface.co/datasets/luizapzbn/from-one-to-many-toxicity-mitigation.thai_toxicity_tweetThai Toxicity Tweet Corpus contains 3,300 tweets annotated by humans with guidelines including a 44-word dictionary.
The author obtained 2,027 and 1,273 toxic and non-toxic tweets, respectively; these were labeled by three annotators. The result of corpus
analysis indicates that tweets that include toxic words are not always toxic. Further, it is more likely that a tweet is toxic, if it contains
toxic words indicating their original meaning. Moreover, disagreements in annotation are primarily because of sarcasm, unclear existing
target, and word sense ambiguity.
Notes from data cleaner: The data is included into [huggingface/datasets](https://www.github.com/huggingface/datasets) in Dec 2020.
By this time, 506 of the tweets are not available publicly anymore. We denote these by `TWEET_NOT_FOUND` in `tweet_text`.
Processing can be found at [this PR](https://github.com/tmu-nlp/ThaiToxicityTweetCorpus/pull/1).ToxicityPromptstask1720_civil_comments_toxicity_classification
Dataset Card for Natural Instructions (https://github.com/allenai/natural-instructions) Task: task1720_civil_comments_toxicity_classification
Additional Information
Citation Information
The following paper introduces the corpus in detail. If you use the corpus in published work, please cite it:
@misc{wang2022supernaturalinstructionsgeneralizationdeclarativeinstructions,
title={Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+… See the full description on the dataset page: https://huggingface.co/datasets/Lots-of-LoRAs/task1720_civil_comments_toxicity_classification.task137_detoxifying-lms_classification_toxicity
Dataset Card for Natural Instructions (https://github.com/allenai/natural-instructions) Task: task137_detoxifying-lms_classification_toxicity
Additional Information
Citation Information
The following paper introduces the corpus in detail. If you use the corpus in published work, please cite it:
@misc{wang2022supernaturalinstructionsgeneralizationdeclarativeinstructions,
title={Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+… See the full description on the dataset page: https://huggingface.co/datasets/Lots-of-LoRAs/task137_detoxifying-lms_classification_toxicity.XSafetyThe dataset is released under the Open Data Commons Attribution License (ODC-By) v1.0 license.
wildguard-trainmultilingual_toxicity_explained
Multilingual and Explainable Toxicity
We explained the toxic part of our multilingual ParaDetox dataset utilizing GPT-4 (May, 2024) with the following prompt:
Please analyze the provided sentence using the structure below to identify elements of
toxicity and suggest improvements, when I tell you, use words from the keywords list (can be
more than one word!):
keywords = [Neutral, Informative, Casual, Assertive, Dismissive, Condescending,
Friendly, Commanding, Instructive… See the full description on the dataset page: https://huggingface.co/datasets/textdetox/multilingual_toxicity_explained.pile-toxicity-balanced3-with-valuespile-chunk-toxicity-scored-3A chunk 3 of the Pile (2.2m documents) scored using the Perspective API (on May 18-20 2022)
pile-toxicity-balanced
Generation procedure
The dataset was constructed using documents from the Pile scored using using Perspective API toxicity scores.
The procedure was the following:
A chunk of the Pile (3%, 7m documents) was scored using the Perspective API.
The first half of this dataset is tomekkorbak/pile-toxic-chunk-0, 100k most toxic documents of the scored chunk
The first half of this dataset is tomekkorbak/pile-nontoxic-chunk-0, 100k least toxic documents of the scored chunk
Then, the dataset… See the full description on the dataset page: https://huggingface.co/datasets/tomekkorbak/pile-toxicity-balanced.toxicity_multilanguage_dataseten_paradetox_toxicity
ParaDetox: Detoxification with Parallel Data (English). Toxicity Task Results
This repository contains information about Toxicity Task markup from English Paradetox dataset collection pipeline.
The original paper "ParaDetox: Detoxification with Parallel Data" was presented at ACL 2022 main conference.
ParaDetox Collection Pipeline
The ParaDetox Dataset collection was done via Yandex.Toloka crowdsource platform. The collection was done in three steps:
Task 1:… See the full description on the dataset page: https://huggingface.co/datasets/s-nlp/en_paradetox_toxicity.DAMO-MultiJailThe dataset is released under the Open Data Commons Attribution License (ODC-By) v1.0 license.
toxicity-multilingual-binary-classification-datasetThis dataset is a comprehensive collection designed to aid in the development of robust and nuanced models for identifying toxic language across multiple languages, while critically distinguishing it from expressions related to mental health, specifically depression. It synthesizes content from three existing public datasets (ToxiGen, TextDetox, and Mental Health - Depression) with a newly generated synthetic dataset (ToxiLLaMA). The creation process involved careful collection, extensive… See the full description on the dataset page: https://huggingface.co/datasets/malexandersalazar/toxicity-multilingual-binary-classification-dataset.combined_toxicity_profanity_v2_train_eval
Dataset Card for "combined_toxicity_profanity_v2_train_eval"
More Information needed
jigsaw-unintended-bias-in-toxicity-classificationhindi-toxicity-qwen-annotations
Annotations for the Hindi-Toxicity classifier 📚
Dataset Summary
This dataset contains the annotations used for training a toxicity classifier (Polygl0t/hindi-roberta-toxicity-classifier). These annotations were generated by Qwen/Qwen2.5-32B-Instruct.
Supported Tasks and Leaderboards
This dataset can be used for the task of text classification, specifically for toxicity detection in Hindi text.
Languages
Hindi.
Dataset Structure… See the full description on the dataset page: https://huggingface.co/datasets/Polygl0t/hindi-toxicity-qwen-annotations.