jeremycochoy/gift-pretrain-full-4096
gift-pretrain-full-4096 Full counterpart to jeremycochoy/gift-pretrain-small-4096: every series of every arrow file in Salesforce/GiftEvalPretrain, cropped into non-overlapping 4096-point windows and globally shuffled. The small companion sub-samples 10 series per sub-dataset; this one keeps everything. 6,376 source arrow files across 152 sub-datasets fully consumed 42,571,692 windows of length 4096 (float32) 4,274 parquet shards, ~619 GB total (zstd) Layout .… See the full description on the dataset page: https://huggingface.co/datasets/jeremycochoy/gift-pretrain-full-4096.
Conversations for this repository live on Hugging Face.
CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.
Open discussions on Hugging Face