Modelpublic

CLMBR/binding-c-command-transformer-0

sourceHugging Faceupdated 3y agoView on Hugging Face

0likes19downloads

tokenizer_config.json13 linesDownload Raw Back to root

1{2  "clean_up_tokenization_spaces": true,3  "eos_token": "<eos>",4  "max_length": null,5  "model_max_length": 512,6  "pad_to_multiple_of": null,7  "pad_token": "<pad>",8  "pad_token_type_id": 0,9  "padding_side": "right",10  "tokenizer_class": "PreTrainedTokenizerFast",11  "unk_token": "<unk>"12}13