Modelpublic

Merlin-Research/HybridIntelligence-0.5B

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

14likes84downloads

tokenizer_config.json14 linesDownload Raw Back to root

1{2  "backend": "tokenizers",3  "clean_up_tokenization_spaces": true,4  "eos_token": "<|end_of_text|>",5  "is_local": true,6  "model_input_names": [7    "input_ids",8    "attention_mask"9  ],10  "model_max_length": 1000000000000000019884624838656,11  "pad_token": "<|end_of_text|>",12  "tokenizer_class": "TokenizersBackend"13}14