CoolFace
Datasetpublic

Beetle-Data/sr-100M-pretok

Beetle-Data/sr-100M-pretok Pretokenized chunks (input_ids = 513-token packed sequences, no cross-document bleeding). Sharded incrementally; the marker data/_finalized.json is committed once all parts are uploaded.

sourceHugging Faceupdated 4mo agoView on Hugging Face
0likes8downloads
Dataset Card

Beetle-Data/sr-100M-pretok

Pretokenized chunks (input_ids = 513-token packed sequences, no cross-document bleeding). Sharded incrementally; the marker data/_finalized.json is committed once all parts are uploaded.