Modelpublic

tiny-random/openai-privacy-filter

sourceHugging Faceupdated 5mo agoView on Hugging Face

1likes174downloads

tokenizer_config.json12 linesDownload Raw Back to root

1{2  "backend": "tokenizers",3  "eos_token": "<|endoftext|>",4  "model_input_names": [5    "input_ids",6    "attention_mask"7  ],8  "model_max_length": 128000,9  "pad_token": "<|endoftext|>",10  "tokenizer_class": "TokenizersBackend"11}12