Modelpublic

LLM-course/simple_tokenizer_retrained2

sourceHugging Facemitupdated 8mo agoView on Hugging Face

0likes22downloads

special_tokens_map.json7 linesDownload Raw Back to root

1{2  "bos_token": "[BOS]",3  "eos_token": "[EOS]",4  "pad_token": "[PAD]",5  "unk_token": "[UNK]"6}7