datatrove
datatrove-testsDatasets used for datatrove testing.
Each split contains the same data:
dst = [
{"text": "hello"},
{"text": "world"},
{"text": "how"},
{"text": "are"},
{"text": "you"},
]
But based on the split name the data are sharded into n-bins
fineweb-350bt-tokenized-datatrove-llama2fineweb-10bt-tokenized-datatrove-llama2s1K-1.1-datatrove
Dataset Card for joelniklaus/s1K-1.1-datatrove
Dataset Summary
Synthetic data generated by DataTrove:
Model: Qwen/Qwen3-0.6B (main)
Source dataset: simplescaling/s1K-1.1 (train split).
Generation config: temperature=0.6, top_p=0.95, top_k=20, max_tokens=8, model_max_context=2048
Speculative decoding: disabled
System prompt: None
User prompt: Column question
The run produced 1,000 samples and generated 8,000 tokens.
You can load the dataset using
from datasets import… See the full description on the dataset page: https://huggingface.co/datasets/joelniklaus/s1K-1.1-datatrove.datatrove-jobs-filter-demodatatrove-test-3-shard