yangwang92/dolma3_mix-150B-1025-merged-gemma4

dolma3_mix-150B-1025-merged (Gemma4) Tokenized copy of the Dolma3 150B mix (dolma3_mix-150B-1025-merged) using the Gemma4 tokenizer. Format Megatron-LM indexed binaries: paired *_text_document.bin and *_text_document.idx files (448 files total, ~616 GiB). These are not Hugging Face datasets Arrow/Parquet shards. Load them with Megatron / Megatron-LM indexed dataset readers. Source name Local / blob name: dolma3_mix-150B-1025-merged-gemma4