Modelpublic

DUTIR-BioNLP/Taiyi-LLM

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face

8likes37downloads

tokenizer_config.json12 linesDownload Raw Back to root

1{2  "auto_map": {3    "AutoTokenizer": [4      "tokenization_qwen.QWenTokenizer",5      null6    ]7  },8  "clean_up_tokenization_spaces": true,9  "model_max_length": 8192,10  "tokenizer_class": "QWenTokenizer"11}12