Modelpublic

FENGYU21/rumor-detection

sourceHugging Faceupdated 4mo agoView on Hugging Face

0likes2downloads

tokenizer_config.json18 linesDownload Raw Back to root

1{
2  "add_prefix_space": false,
3  "backend": "tokenizers",
4  "bos_token": "<s>",
5  "cls_token": "<s>",
6  "eos_token": "</s>",
7  "errors": "replace",
8  "is_local": false,
9  "local_files_only": false,
10  "mask_token": "<mask>",
11  "model_max_length": 1000000000000000019884624838656,
12  "pad_token": "<pad>",
13  "sep_token": "</s>",
14  "tokenizer_class": "RobertaTokenizer",
15  "trim_offsets": true,
16  "unk_token": "<unk>"
17}
18