LH-Tech-AI/htmLLM-50M-Base

1---2license: apache-2.03datasets:4- bigcode/the-stack-smol5language:6- en7tags:8- code9- base10- coding11- small12- tiny13pipeline_tag: text-generation14new_version: LH-Tech-AI/htmLLM-124M15---16 17# Welcome to htmLLM 50M Base!18 19With this LLM, we wanted to see, how well tiny LLMs with just 50 million parameters can perform on coding tasks.20 21If you want to try it, you can use htmllm.ipynb in the HF model files and download the model weight from this HF model.22 23# Code24All code can be accessed via the file **htmllm.ipynb** in this HF model.25 26# Weights27The final **base** model checkpoint can be downloaded here in the files list as **ckpt.pt**.28 29# Training30We trained our model on a single Kaggle T4 GPU.31 32# Thanks to:33- Andrej Karpathy and his nanoGPT code34- Kaggle for the free GPU hours for training on the T435- You all for your support on my reddit.