Modelpublic

miguelcsx/causal-focus-bbpe16k-tokenizer

sourceHugging Faceotherupdated 3mo agoView on Hugging Face

0likes

Model Card

CausalFocus BBPE16K Tokenizer

Tokenizer-only repository for CausalFocus-GPTBERT-D256-BBPE16K.

—family: byte-level BPE
—vocab size: 16000
—max sequence length used by release training: 512
—source corpus: miguelcsx/babylm-corpus-cleaned