Apppublic

DanielNonStop/TextProcessing

sourceHugging Faceupdated 4y agoView on Hugging Face

0likes

special_tokens_map.json20 linesDownload Raw Back to root

1{2  "additional_special_tokens": [3    "<eop>",4    "<eod>"5  ],6  "bos_token": "<s>",7  "cls_token": "<cls>",8  "eos_token": "</s>",9  "mask_token": {10    "content": "<mask>",11    "lstrip": true,12    "normalized": false,13    "rstrip": false,14    "single_word": false15  },16  "pad_token": "<pad>",17  "sep_token": "<sep>",18  "unk_token": "<unk>"19}20