Modelpublic

aixk/fastplus-40m

sourceHugging Faceupdated 3mo agoView on Hugging Face

0likes

tokenizer_config.json14 linesDownload Raw Back to root

1{2  "backend": "tokenizers",3  "bos_token": "<s>",4  "eos_token": "</s>",5  "extra_special_tokens": [6    "<|user|>",7    "<|assistant|>"8  ],9  "model_max_length": 1000000000000000019884624838656,10  "pad_token": "<pad>",11  "tokenizer_class": "TokenizersBackend",12  "unk_token": "<unk>"13}14