Modelpublic

xianf/testmodel_2

sourceHugging Facebigscience-openrail-mupdated 3y agoView on Hugging Face

0likes119downloads

tokenizer_config.json13 linesDownload Raw Back to root

1{2  "additional_special_tokens": null,3  "clean_up_tokenization_spaces": true,4  "eos_token": "</s>",5  "extra_ids": 0,6  "legacy": true,7  "model_max_length": 1000000000000000019884624838656,8  "pad_token": "<pad>",9  "sp_model_kwargs": {},10  "tokenizer_class": "T5Tokenizer",11  "unk_token": "<unk>"12}13