Koushim/processed-jigsaw-toxic-comments
Processed Jigsaw Toxic Comments Dataset This is a preprocessed and tokenized version of the original Jigsaw Toxic Comment Classification Challenge dataset, prepared for multi-label toxicity classification using transformer-based models like BERT. โ ๏ธ Important Note: I am not the original creator of the dataset. This dataset is a cleaned and restructured version made for quick use in PyTorch deep learning models. ๐ฆ Dataset Features Each example contains: text:โฆ See the full description on the dataset page: https://huggingface.co/datasets/Koushim/processed-jigsaw-toxic-comments.
Processed Jigsaw Toxic Comments Dataset
This is a preprocessed and tokenized version of the original Jigsaw Toxic Comment Classification Challenge dataset, prepared for multi-label toxicity classification using transformer-based models like BERT.
โ ๏ธ Important Note: I am not the original creator of the dataset. This dataset is a cleaned and restructured version made for quick use in PyTorch deep learning models.
๐ฆ Dataset Features
Each example contains:
text: The original user commentlabels: A list of 7 binary float values indicating presence of toxicity categoriesinput_ids,attention_mask: Tokenized fields usingbert-base-uncased(max length 128)
Toxicity Categories:
toxicitysevere_toxicityobscenethreatinsultidentity_attacksexual_explicit
๐งช Dataset Splits
๐ง Processing Details
- Original Source: Manually downloaded from Kaggle
- Preprocessing:
- Combined multiple toxicity columns into a single
labelsvector - Converted label values to floats (0.0 or 1.0)
- Tokenization:
- Used Hugging Face
bert-base-uncasedtokenizer - Applied padding and truncation to max length of 128
- Formatting:
- Final dataset set to return PyTorch
input_ids,attention_mask, andlabels
๐ก Usage Example
from datasets import load_dataset
dataset = load_dataset("Koushim/processed-jigsaw-toxic-comments")
from torch.utils.data import DataLoader
train_loader = DataLoader(dataset["train"], batch_size=32, shuffle=True)
batch = next(iter(train_loader))
print(batch['input_ids'].shape) # torch.Size([32, 128])
print(batch['labels'].shape) # torch.Size([32, 7])๐ Citation
If you use this dataset, please cite the original Jigsaw authors:
@misc{jigsawtoxic,
title={Toxic Comment Classification Challenge},
author={Jigsaw and Google},
year={2018},
url={https://www.kaggle.com/c/jigsaw-toxic-comment-classification-challenge}
}๐ Acknowledgements
- Original dataset by Jigsaw/Google
- Processing, formatting, and tokenization by Koushik
