datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Gutenberg-Clean
📚 TinyWay-Gutenberg-Clean (Compressed Shards)
A large-scale, high-quality English text dataset derived from Project Gutenberg.
The corpus has been cleaned, normalized, deduplicated, segmented into fixed-length samples, and stored as compressed JSONL shards for efficient large-scale language model training.
This dataset is intended for pretraining and experimentation with small and medium language models such as TinyWay, tokenizer training, and large-scale NLP research.… See the full description on the dataset page: https://huggingface.co/datasets/NNEngine/Gutenberg-Clean.gutenberg-txt-frThis dataset is not yet refined enough to do other things than tests, it contain a mix of french and english and special caracters (french accentuation).
Dataset Card for gutenberg-txt-fr
218 french books, txt file format, from Gutenberg project.
Dataset Sources
Gutemberg project : https://www.gutenberg.org
Uses
Could be used for RAG, datas are not labeled.
Why ?
I needed a dataset of txt files to test RAG pipelines, didn't find one suitable… See the full description on the dataset page: https://huggingface.co/datasets/cabusar/gutenberg-txt-fr.gutenberg-txt-de
Dataset Card for gutenberg-txt-fr
93 deutch books, txt file format, from Gutenberg project.
Dataset Sources
Gutemberg project : https://www.gutenberg.org
Uses
Could be used for RAG, datas are not labeled.
Why ?
I needed a dataset of txt files to test RAG pipelines, didn't find one suitable, created this one, shared it. :)
gutenberg_selected_ebooks
Gutenberg selected ebooks dataset
This dataset is a collection of passages from ebooks handpicked from the Gutenberg Project.
These writings are:
Alice's Adventures in Wonderland
Pride and Prejudice
Romeo and Juliet
The Adventures of Sherlock Holmes
The Odyssey
Winnie-the-Pooh
Source
The texts of the passages were derived from a larger Gutenberg-based set: sedthh/gutenberg_english, which was sourced directly from the project's site.
Metadata
Each passage… See the full description on the dataset page: https://huggingface.co/datasets/enelpol/gutenberg_selected_ebooks.