HCAI-Lab-GT/dolma3-6t-sample-100000-docs
dolma3-6t-sample-100000-docs Materialized stratified sample of 100,000 docs per bin (~58k source-shard files, ~186 GB) from the deduplicated 6T Dolma3 corpus. Seed 42, materialized by 128 Modal workers (worker_0000/ through worker_0127/). Layout HCAI-Lab/dolma3-6t-sample-100000-docs/ ├── bin_summary.csv ├── sample_contract.json └── worker_NNNN/ └── soc127__phase1_pool_shared__...__shard_NNNNNNNN.jsonl.zst (~450 files per worker) Dual-access… See the full description on the dataset page: https://huggingface.co/datasets/HCAI-Lab-GT/dolma3-6t-sample-100000-docs.
Add README documenting bucket/dataset dual-access
Add root metadata files
Add worker_0127
Add worker_0126
Add worker_0125
Add worker_0124
Add worker_0123
Add worker_0122
Add worker_0121
Add worker_0120
Add worker_0119
Add worker_0118
Add worker_0117
Add worker_0116
Add worker_0115
Add worker_0114
Add worker_0113
Add worker_0112
Add worker_0111
Add worker_0110
Add worker_0109
Add worker_0108
Add worker_0107
Add worker_0106
Add worker_0105
Add worker_0104
Add worker_0103
Add worker_0102
Add worker_0101
Add worker_0100
Add worker_0099
Add worker_0098
Add worker_0097
Add worker_0096
Add worker_0095
Add worker_0094
Add worker_0093
Add worker_0092
Add worker_0091
Add worker_0090
Add worker_0089
Add worker_0088
Add worker_0087
Add worker_0086
Add worker_0085
Add worker_0084
Add worker_0083
Add worker_0082
Add worker_0081
Add worker_0080