Modelpublic

zai-org/WebGLM

sourceHugging Faceupdated 3y agoView on Hugging Face

48likes226downloads

tokenizer_config.json19 linesDownload Raw Back to root

1{2  "name_or_path": "THUDM/glm-10b",3  "eos_token": "<|endoftext|>",4  "pad_token": "<|endoftext|>",5  "cls_token": "[CLS]",6  "mask_token": "[MASK]",7  "unk_token": "[UNK]",8  "additional_special_tokens": ["<|startofpiece|>", "<|endofpiece|>", "[gMASK]", "[sMASK]"],9  "add_prefix_space": false,10  "tokenizer_class": "GLMGPT2Tokenizer",11  "use_fast": false,12  "auto_map": {13    "AutoTokenizer": [14      "tokenization_glm.GLMGPT2Tokenizer",15      null16      ]17  }18}19