Modelpublic

ProCreations/Booper-Big-Chat

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face

1likes167downloads

tokenizer_config.json15 linesDownload Raw Back to root

1{2  "backend": "tokenizers",3  "bos_token": "<bos>",4  "clean_up_tokenization_spaces": false,5  "eos_token": "<eos>",6  "is_local": true,7  "local_files_only": false,8  "model_max_length": 4096,9  "pad_token": "<pad>",10  "padding_side": "right",11  "sep_token": "<sep>",12  "tokenizer_class": "TokenizersBackend",13  "truncation_side": "right"14}15