joshycodes/op-spp-streams-v1
op-spp-streams-v1 — tokenized Megatron streams (SPP-format pretraining corpus) Tokenized Dolma v1.7 (ODC-BY) subsample in Megatron IndexedDataset format (uint16, SmolLM2 tokenizer + <assistant> extension from epfl-dlab/spp-training): compact dense-packed 2049-token windows; annotated/canary one document per window, EOD-padded. Built for a Synthetic-Persona-Pretraining-recipe run (arXiv:2608.13482) — these files carry ONLY document tokens (raw public-corpus text); the persona… See the full description on the dataset page: https://huggingface.co/datasets/joshycodes/op-spp-streams-v1.
Conversations for this repository live on Hugging Face.
CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.
Open discussions on Hugging Face