Modelpublic

physicsrob/torchwright-doom-e1m1

sourceHugging Faceupdated 2mo agoView on Hugging Face

3likes169downloads

tokenizer_config.json15 linesDownload Raw Back to root

1{2  "backend": "tokenizers",3  "bos_token": "bos",4  "clean_up_tokenization_spaces": false,5  "eos_token": "done",6  "model_input_names": [7    "input_ids",8    "attention_mask"9  ],10  "model_max_length": 1000000000000000019884624838656,11  "pad_token": "done",12  "tokenizer_class": "TokenizersBackend",13  "unk_token": "<unk>"14}15