Datasetpublic

Efe2898/Prosperity-Family-Alya-CPT-3B-Kumru

Prosperity Family Alya CPT 3B — Kumru Tokenized Status: finished - 2B remain - 1B Developer: Prosperity AIModel/tokenizer: Efe2898/Prosperity-Family-Alya-BaseSource dataset: moganai/turkishfineweb2-cleaned Tokenizer Revision: df12a3c9e14c80d1464a8d7a2d624b2b021ab283Vocabulary: 50,176EOS token ID: 3 Filtering language_score >= 0.98 fasttext_clean_score >= 0.75 document chars: 200 .. 500000 deterministic stream shuffle seed: 20260906 shuffle buffer:… See the full description on the dataset page: https://huggingface.co/datasets/Efe2898/Prosperity-Family-Alya-CPT-3B-Kumru.

sourceHugging Faceodc-byupdated 20d agoView on Hugging Face

0likes129downloads

Dataset Card

Prosperity Family Alya CPT 3B — Kumru Tokenized

Status: finished - 2B remain - 1B

Developer: Prosperity AI Model/tokenizer: Efe2898/Prosperity-Family-Alya-Base Source dataset: moganai/turkishfineweb2-cleaned

Tokenizer

Revision: df12a3c9e14c80d1464a8d7a2d624b2b021ab283 Vocabulary: 50,176 EOS token ID: 3

Filtering

—language_score >= 0.98
—fasttext_clean_score >= 0.75
—document chars: 200 .. 500000
—deterministic stream shuffle seed: 20260906
—shuffle buffer: 5,000

Storage

—continuous token stream
—little-endian uint16
—EOS between source documents
—no fixed training sequence length
—no binary header
—target shard size: about 100,000,000 tokens

Target

Exactly 3,000,000,000 tokens.

Raw token body: 6,000,000,000 bytes.

Current progress

Committed tokens: 0 Source docs committed: 0 Accepted docs: 0 Uploaded shards: 0