stanford-crfm/heuristic_classification-filtered-pile-50M

Dataset Card for heuristic_classification-filtered-pile-50M Dataset Summary This dataset is a subset of The Pile, selected via the heuristic classification data selection method. The target distribution for heuristic classification are the Wikipedia and BookCorpus2 subsets of The Pile. Languages English (EN) Dataset Structure A train set is provided (51.2M examples) in jsonl format. Data Instances {"contents": "Members… See the full description on the dataset page: https://huggingface.co/datasets/stanford-crfm/heuristic_classification-filtered-pile-50M.