kaushik-harsh-99/Uncensored-SFT-v1

Dataset Creation Process

This dataset was not scraped from a single source.

Instead, it was built through a large multi-stage curation and cleaning pipeline involving many open instruction datasets available on Hugging Face.

The entire dataset was normalized into a unified:

{
  "input": "...",
  "output": "..."
}

format.

Data Collection

A large number of public instruction datasets were downloaded from Hugging Face.

These datasets included:

• —Instruction-following datasets
• —Chat datasets
• —QA datasets
• —Roleplay datasets
• —Coding datasets
• —General assistant conversations
• —Uncensored datasets
• —Open conversational corpora

Many datasets originally existed in different formats such as:

• —Alpaca format
• —ShareGPT format
• —ChatML format
• —Message-based structures
• —Instruction/input/output triples
• —Raw JSONL
• —Arrow datasets

All datasets were converted into a single standardized schema.

Dataset Processing Pipeline

1. Format Normalization

All datasets were normalized into a common structure:

{
  "input": "...",
  "output": "..."
}

This involved:

• —Removing system prompts
• —Removing metadata
• —Flattening multi-message chat structures
• —Extracting user-assistant pairs
• —Converting multiple schemas into one consistent format

2. OCR and Junk Cleaning

Basic OCR and text corruption cleaning was performed.

This included removal of:

• —Broken Unicode characters
• —OCR artifacts
• —Repeated symbol junk
• —Corrupted whitespace
• —Encoding issues
• —Extremely malformed rows
• —Random garbage strings

The cleaning pipeline was intentionally conservative to avoid damaging useful training data.

3. English-Only Filtering

The dataset was aggressively filtered to retain only English samples.

This involved:

• —Unicode script detection
• —Language detection
• —Removal of multilingual rows
• —Removal of non-English scripts

Languages/scripts filtered included:

• —Chinese
• —Japanese
• —Korean
• —Hindi
• —Arabic
• —Russian/Cyrillic
• —Bengali
• —Tamil
• —Telugu
• —Gujarati
• —Punjabi
• —Hebrew
• —Greek
• —Thai
• —Burmese
• —Tibetan
• —and many others

Only high-confidence English samples were retained.

4. Exact Deduplication

Exact duplicate rows were removed.

This step eliminated:

• —Repeated instruction-response pairs
• —Re-uploaded datasets
• —Duplicate synthetic generations
• —Mirrored samples

This significantly reduced redundancy and improved information density.

5. Question Deduplication

Many open instruction datasets contain repeated or near-identical questions with different responses.

The dataset was deduplicated at the question level using normalized input matching.

Examples:

What is AI?
what is ai
What is AI!!!

These were treated as identical questions.

6. Best Response Selection

When multiple responses existed for the same question, the pipeline retained the highest-quality candidate.

Selection criteria included:

• —Response length
• —Information density
• —Completeness
• —Reduced redundancy
• —Better formatting

The longest and most informative response was generally retained.

7. Low-Quality Filtering

Additional filtering was performed to remove:

• —Extremely short responses
• —Empty outputs
• —Corrupted samples
• —Low-information generations
• —Formatting failures
• —Broken rows

This improved overall dataset quality and reduced noisy supervision.

Final Dataset Statistics

Example Format

{
  "input": "Explain what a black hole is.",
  "output": "A black hole is a region of spacetime where gravity is so strong that nothing, including light, can escape from it..."
}

Intended Use

This dataset is intended for:

• —Supervised fine-tuning
• —Instruction tuning
• —Chat model training
• —Alignment experimentation
• —Uncensored assistant research
• —Synthetic data experiments
• —Small model capability improvement

Potential applications:

• —LLaMA fine-tuning
• —Mistral fine-tuning
• —Qwen fine-tuning
• —Gemma fine-tuning
• —Distillation
• —Reasoning experiments
• —Alignment studies

Recommended Training Usage

Recommended for:

• —QLoRA
• —Full fine-tuning
• —Continued pretraining
• —Distillation
• —SFT
• —Alignment tuning

Works particularly well for:

• —Smaller models
• —Instruction-following recovery
• —Reducing over-refusal behavior
• —Increasing response completeness

File Structure

train.jsonl

Each line is a standalone JSON object.

Acknowledgements

Thanks to the open-source AI community and all original dataset contributors on Hugging Face whose datasets made this aggregation possible.