sanjuhs/ternary-tinystories-4096
TinyStories 4K-BPE token stream This is a deterministic, tokenized derivative used by the Ternary LLM Experiment. The source is roneneldan/TinyStories. tokenizer: byte-level BPE, vocabulary 4,096 token dtype: little-endian uint16 training stories: 2,119,719 training tokens: 488,174,163 validation stories: 21,990 validation tokens: 4,907,807 Each story is encoded with explicit beginning/end markers and concatenated into train.bin or validation.bin. metadata.json records the… See the full description on the dataset page: https://huggingface.co/datasets/sanjuhs/ternary-tinystories-4096.
Conversations for this repository live on Hugging Face.
CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.
Open discussions on Hugging Face