devoppro/gpt-python-124m

1---2tags:3- gpt4- from-scratch5- code-generation6---7 8# gpt-python-124m9 10A ~124M parameter GPT-2-style decoder-only transformer, trained from scratch on the11[devoppro/Python](https://huggingface.co/datasets/devoppro/Python) dataset.12 13- Layers: 1214- Heads: 1215- Embedding dim: 76816- Context length: 51217- Trained to step 0, val loss 10.390418- Tokenizer: GPT-2 BPE (tiktoken 'gpt2')19