AllanK24/openai-moderation-binary

🧠 OpenAI Moderation Binary Dataset

This dataset is a binary-labeled version of the original OpenAI Moderation Evaluation Dataset, created to support safe/unsafe classification tasks in content moderation, safety research, and AI alignment.

📦 Dataset Details

• —Original Source: OpenAI Moderation API Evaluation Dataset
• —License: MIT (inherited from original repo)
• —Samples: 1,680 total
• —Labels:
• —"safe" (no harm labels present)
• —"unsafe" (at least one moderation label present)

📁 Structure

Each row consists of:

{
  "prompt": "Some user input text...",
  "prompt_label": "safe"  // or "unsafe"
}

🧹 Preprocessing

This version was derived by:

1. 1.Downloading and parsing the original JSONL dataset (samples-1680.jsonl.gz)
2. 2.Creating a new column called prompt_label, based on the presence of any of the following 8 moderation labels:
3. 3.S (sexual)
4. 4.S3 (severe sexual)
5. 5.H (hate)
6. 6.H2 (severe hate)
7. 7.V (violence)
8. 8.V2 (severe violence)
9. 9.HR (harassment)
10. 10.SH (self-harm)
11. 11.Assigning:
12. 12.prompt_label = "unsafe" if any of those were 1
13. 13.prompt_label = "safe" if all were 0
14. 14.Removing the original moderation columns, leaving only:
15. 15.prompt
16. 16.prompt_label

📊 Label Distribution

💡 Intended Use

This dataset is designed for:

• —Binary classification (safe vs unsafe prompt detection)
• —Content moderation and safety evaluation
• —Educational and research purposes

📚 Citation

If you use this dataset, please cite the original authors of the OpenAI Moderation dataset:

OpenAI (2022). A Holistic Approach to Undesired Content Detection in the Real World. https://github.com/openai/moderation-api-release

🙏 Acknowledgements

Huge credit to OpenAI for releasing the original dataset.

This binary-labeled version was created for ease of evaluation and validation.