Efe2898/Prosperity-Family-Alya-CPT-3B-Kumru
Prosperity Family Alya CPT 3B — Kumru Tokenized Status: finished - 2B remain - 1B Developer: Prosperity AIModel/tokenizer: Efe2898/Prosperity-Family-Alya-BaseSource dataset: moganai/turkishfineweb2-cleaned Tokenizer Revision: df12a3c9e14c80d1464a8d7a2d624b2b021ab283Vocabulary: 50,176EOS token ID: 3 Filtering language_score >= 0.98 fasttext_clean_score >= 0.75 document chars: 200 .. 500000 deterministic stream shuffle seed: 20260906 shuffle buffer:… See the full description on the dataset page: https://huggingface.co/datasets/Efe2898/Prosperity-Family-Alya-CPT-3B-Kumru.
we cannot reach target bc kaggle's limits. anyways, this dataset ready to train.
Add shard 00022: 2,299,979,089/3,000,000,000 tokens
Add shard 00021: 2,199,979,922/3,000,000,000 tokens
Add shard 00020: 2,099,980,399/3,000,000,000 tokens
Add shard 00019: 1,999,980,742/3,000,000,000 tokens
Add shard 00018: 1,899,980,810/3,000,000,000 tokens
Add shard 00017: 1,799,981,078/3,000,000,000 tokens
Add shard 00016: 1,699,981,464/3,000,000,000 tokens
Add shard 00015: 1,599,981,693/3,000,000,000 tokens
Add shard 00014: 1,499,983,768/3,000,000,000 tokens
Add shard 00013: 1,399,983,941/3,000,000,000 tokens
Add shard 00012: 1,299,984,062/3,000,000,000 tokens
Add shard 00011: 1,199,984,135/3,000,000,000 tokens
Add shard 00010: 1,099,984,435/3,000,000,000 tokens
Add shard 00009: 999,984,540/3,000,000,000 tokens
Add shard 00008: 899,984,667/3,000,000,000 tokens
Add shard 00007: 799,985,090/3,000,000,000 tokens
Add shard 00006: 699,985,732/3,000,000,000 tokens
Add shard 00005: 599,987,816/3,000,000,000 tokens
Add shard 00004: 499,988,410/3,000,000,000 tokens
Add shard 00003: 399,988,436/3,000,000,000 tokens
Add shard 00002: 299,992,782/3,000,000,000 tokens
Add shard 00001: 199,997,769/3,000,000,000 tokens
Add shard 00000: 99,998,715/3,000,000,000 tokens
Initialize Alya CPT 3B dataset
initial commit