Modelpublic

Nitin98/Datasets

sourceHugging Facemitupdated 2y agoView on Hugging Face

0likes12downloads

rootPythonCodeGeneratorspecial_tokens_map.json

special_tokens_map.json25 linesDownload Raw Back to PythonCodeGenerator

1{2  "bos_token": {3    "content": "<|endoftext|>",4    "lstrip": false,5    "normalized": false,6    "rstrip": false,7    "single_word": false8  },9  "eos_token": {10    "content": "<|endoftext|>",11    "lstrip": false,12    "normalized": false,13    "rstrip": false,14    "single_word": false15  },16  "pad_token": "<|endoftext|>",17  "unk_token": {18    "content": "<|endoftext|>",19    "lstrip": false,20    "normalized": false,21    "rstrip": false,22    "single_word": false23  }24}25