codebyam/SmallLM

1---2license: mit3language:4- en5tags:6- code7- model8- slm9- model10- language model11pipeline_tag: text-generation12library_name: PyTorch13---14 15 16 17# SmallLM18 19SmallLM is a specialized transformer-based language model with fewer than 50 million parameters. It utilizes a tokenizer derived from minbpe, developed by Andrej Karpathy, and has been trained on custom datasets. This model is currently in active development, and we are continuously working to enhance its performance. The model can be used on any small end devices without any issue. The model has not been released yet, but we are working to make it available soon. Stay tuned!20 21## Features22 23- Can be specialized for specific tasks24- Compact model size (less than 50M parameters)25- Custom tokenizer from minbpe26- Trained on custom datasets27- Can run CPU based devices28 29## Limitations30 31Due to its relatively small size, limited training data, and shorter training duration, the model may occasionally underperform compared to small language models.32 33## Contact34 35For any questions or feedback, please reach out to us through Discussions.36 37## Resources38- Tokenizer Source: <a href="https://github.com/karpathy/minbpe">minbpe</a>39 40