raulmodena/leire-corpus

Leire Corpus (EN) Tokenized pretraining corpus for Leire, a 343.7M-parameter Brazilian Portuguese LM trained from scratch on Kaggle T4s. ~15B tokens, 70% PT / 15% code / 8% math / 7% educational English, tokenized with a custom 32,768 BPE vocabulary trained on the same mixture. Shards are uint16 binaries; recipe and stats below (in Portuguese). Corpus de pre-treino da Leire, um LM de 343,7M de parametros em portugues brasileiro, treinado do zero em T4 do Kaggle. O projeto e… See the full description on the dataset page: https://huggingface.co/datasets/raulmodena/leire-corpus.