CoolFace
Datasetpublic

openeurollm/dolci-think-sft-tokenized

Dolci-Think-SFT Tokenized Pre-tokenized version of the allenai/Dolci-Think-SFT-7B dataset, ready for training with OLMo-core. This dataset was used to train the openeurollm/OLMo-3-7B-Think-SFT checkpoints. See also: openeurollm/dolci-instruct-sft-tokenized for the instruct (non-thinking) variant. Dataset Details Property Value Source dataset allenai/Dolci-Think-SFT-7B Tokenizer allenai/Olmo-3-7B-Think-SFT Max sequence length 32,768 Total instances… See the full description on the dataset page: https://huggingface.co/datasets/openeurollm/dolci-think-sft-tokenized.

sourceHugging Faceapache-2.0updated 7mo agoView on Hugging Face
0likes159downloads
Dataset Card

Dolci-Think-SFT Tokenized

Pre-tokenized version of the allenai/Dolci-Think-SFT-7B dataset, ready for training with OLMo-core.

This dataset was used to train the openeurollm/OLMo-3-7B-Think-SFT checkpoints.

See also: openeurollm/dolci-instruct-sft-tokenized for the instruct (non-thinking) variant.

Dataset Details

PropertyValue
Source datasetallenai/Dolci-Think-SFT-7B
Tokenizerallenai/Olmo-3-7B-Think-SFT
Max sequence length32,768
Total instances2,268,177
Total tokens22.7B
Trainable tokens22.2B (97.5%)
Avg tokens per instance10,018

During SFT, only assistant response tokens are trainable. System and user message tokens are masked out via labels_mask so the model sees them as context but is not trained to predict them. The high trainable ratio (97.5%) reflects the long chain-of-thought responses in this dataset, where the assistant reasoning dominates each sequence.

File Format

The dataset is stored as pre-merged NumPy arrays compatible with OLMo-core's data loading:

  • —token_ids_part_XXXX.npy: token ID arrays (84 parts)
  • —labels_mask_part_XXXX.npy: label mask arrays (84 parts), where 1 = trainable (assistant response) and 0 = masked (system/user message)
  • —tokenizer/: tokenizer files used during tokenization
  • —dataset_statistics.json: detailed statistics about the tokenized dataset

Usage with OLMo-core

Point your OLMo-core training config to this dataset directory. The format is directly compatible with the OLMo-core SFT data loader.

License

Apache 2.0