nandhakumarms/qualc-fineweb-edu-en

QualC FineWeb-Edu English (Cleaned)

QualC FineWeb-Edu English (Cleaned) is a cleaned subset of the official FineWeb-Edu dataset published by Hugging Face.

The dataset is intended for Large Language Model (LLM) pretraining, tokenizer training, continual pretraining, educational NLP research, and language modeling.

This repository contains approximately one million cleaned educational English documents prepared for the QualC project.

Dataset Information

Source

This dataset is derived from the official HuggingFaceFW/fineweb-edu dataset.

FineWeb-Edu is a large-scale educational web corpus built from Common Crawl. Educational pages are selected using an educational quality classifier designed specifically for LLM pretraining.

Original Dataset

HuggingFaceFW/fineweb-edu

Cleaning Pipeline

The following preprocessing steps were applied:

• —Removed empty documents
• —Removed HTML tags
• —Removed invalid Unicode control characters
• —Fixed Unicode replacement characters (�)
• —Applied Unicode NFKC normalization
• —Normalized whitespace
• —Removed duplicate documents within each processing batch

URLs were intentionally retained because they are part of natural educational web content.

Validation

Before Cleaning

After Cleaning

Dataset Structure

Feature:

Example

{
  "text": "Educational English document..."
}

Intended Uses

This dataset is suitable for:

• —Large Language Model Pretraining
• —Language Modeling
• —Tokenizer Training
• —Continued Pretraining
• —Educational NLP
• —Retrieval Systems
• —Academic Research
• —Text Generation

Limitations

Although cleaned, this dataset still reflects naturally occurring educational web pages.

Some documents may contain:

• —URLs
• —Educational references
• —Product names
• —Organization names
• —Website navigation text

Additional filtering may be required depending on downstream applications.

License

This dataset is derived from the official FineWeb-Edu dataset.

Original License:

Open Data Commons Attribution License (ODC-By) v1.0

Use of the original data is also subject to the Common Crawl Terms of Use.

Attribution

Please acknowledge both this repository and the original FineWeb-Edu project when using this dataset for research or model training.

Original Dataset:

HuggingFaceFW/fineweb-edu

Citation

If you use this dataset, please cite the FineWeb paper.

@inproceedings{penedo2024fineweb,
  title={The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale},
  author={Guilherme Penedo and Hynek Kydlíček and Loubna Ben Allal and Anton Lozhkov and Margaret Mitchell and Colin Raffel and Leandro von Werra and Thomas Wolf},
  booktitle={NeurIPS Datasets and Benchmarks},
  year={2024}
}

Maintainer

Nandhakumar M.S

Project:

QualC

QualC is an open dataset collection project focused on building clean, high-quality datasets for training open Large Language Models.

Acknowledgements

This dataset is a cleaned derivative of the official FineWeb-Edu dataset released by the Hugging Face team.

Credit goes to the FineWeb authors for creating and maintaining the original dataset.