harims95/hobbylm-longctx-2p5b
HobbyLM-1B Long-Context Extension Corpus 2.5B GPT-2 tokens staged for extending harims95/hobbylm-1b-hf's context window from 1024 tokens to 4096, then 8192. Format Each <source>_NNNNNN.bin file is the same flat-binary format used throughout HobbyLM training: a 256 x int32 header (magic=20240520, version=1, num_tokens), followed by the token stream as uint16 GPT-2 token ids. The flat .bin files do not encode document boundaries on their own. Each source also ships… See the full description on the dataset page: https://huggingface.co/datasets/harims95/hobbylm-longctx-2p5b.
Fix replay source: correct EOT-boundary document indexing (was 39 coarse pseudo-docs with corrupted offsets, now 281,571 real documents)
Fix replay source: correct EOT-boundary document indexing (was 39 coarse pseudo-docs with corrupted offsets, now 281,571 real documents)
Fix replay source: correct EOT-boundary document indexing (was 39 coarse pseudo-docs with corrupted offsets, now 281,571 real documents)
Fix replay source: correct EOT-boundary document indexing (was 39 coarse pseudo-docs with corrupted offsets, now 281,571 real documents)
Upload HobbyLM-1B long-context corpus: 2.5B tokens, 6 sources, block-diagonal packing support (part 2)
Upload HobbyLM-1B long-context corpus: 2.5B tokens, 6 sources, block-diagonal packing support
initial commit