timaeus/dsir-pile-13m-filtered-no-github-or-dm_mathematics

DSIR Pile 13M - Filtered Version This is a filtered version of timaeus/dsir-pile-13m. Filtering Applied: Excluded: All rows where metadata.pile_set_name contains 'Github' or 'DM_mathematics' Kept: All other rows from the original dataset Dataset Size Original: ~13M examples Filtered: 12,782,200 examples (99.9% of original) Uploaded in: 64 batch files Usage from datasets import load_dataset dataset =… See the full description on the dataset page: https://huggingface.co/datasets/timaeus/dsir-pile-13m-filtered-no-github-or-dm_mathematics.