Datasetpublic

placeholderlabs/pretrain-nemotron-math-mix

Normalized documents plus aligned Dolma-2 tokens and target masks. Size Tokens 22,927,812,461 (22.9B) Trainable tokens 22,927,812,461 (22.9B) Documents 21,377,358 Shards 180 UTF-8 bytes 77,994,866,327 Tokenizer allenai/dolma2-tokenizer@5292e5d6c0f4 documents.parquet - document_id, text, part_ends, part_trainable, must_not_split. The readable payload and the mask intent. metadata.parquet - one text-free row per document: token span, source, stratum… See the full description on the dataset page: https://huggingface.co/datasets/placeholderlabs/pretrain-nemotron-math-mix.

sourceHugging Faceupdated 14d agoView on Hugging Face

0likes132downloads

Dataset Card

Normalized documents plus aligned Dolma-2 tokens and target masks.

Size


Tokens	22,927,812,461 (22.9B)
Trainable tokens	22,927,812,461 (22.9B)
Documents	21,377,358
Shards	180
UTF-8 bytes	77,994,866,327
Tokenizer	`allenai/dolma2-tokenizer@5292e5d6c0f4`

—documents.parquet - document_id, text, part_ends, part_trainable, must_not_split. The readable payload and the mask intent.
—metadata.parquet - one text-free row per document: token span, source, stratum, sizes, provenance and the full source metadata.
—tokens.bin / offsets.bin / target-mask.bin - the training hot path. Little-endian int32 IDs, int64 document boundaries, one LSB-first mask bit per token. RSDB packs 4K or 16K sequences at training time.