Modelpublic

AxionLab-official/MiniBot-0.9M-Base

sourceHugging Facemitupdated 2d agoView on Hugging Face

0likes242downloads

tokenizer_config.json11 linesDownload Raw Back to root

1{
2  "backend": "tokenizers",
3  "bos_token": "<bos>",
4  "eos_token": "<eos>",
5  "is_local": true,
6  "model_max_length": 128,
7  "pad_token": "<pad>",
8  "tokenizer_class": "TokenizersBackend",
9  "unk_token": "<unk>"
10}
11