NNEngine/Gutenberg-Clean
📚 TinyWay-Gutenberg-Clean (Compressed Shards) A large-scale, high-quality English text dataset derived from Project Gutenberg. The corpus has been cleaned, normalized, deduplicated, segmented into fixed-length samples, and stored as compressed JSONL shards for efficient large-scale language model training. This dataset is intended for pretraining and experimentation with small and medium language models such as TinyWay, tokenizer training, and large-scale NLP research.… See the full description on the dataset page: https://huggingface.co/datasets/NNEngine/Gutenberg-Clean.
Conversations for this repository live on Hugging Face.
CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.
Open discussions on Hugging Face