nishantup/nanogpt-slm-tinystories-instruct
053
nanoGPT SLM TinyStories Instruct -- 124.0M Parameters
An instruction-tuned Small Language Model trained entirely from scratch:
- Pretrained on TinyStories (2.1M children's stories, 70K iterations)
- Instruction fine-tuned (SFT) on a 300K multi-source instruction dataset
What This Model Does
This model follows instructions across diverse tasks: answering questions, summarizing text, creative writing, classification, translation, and more. Give it a task and it responds:
Task: Explain what photosynthesis is in simple terms.
Answer:
Photosynthesis is the process by which plants convert sunlight, water, and
carbon dioxide into glucose and oxygen. It occurs in the chloroplasts of
plant cells and is essential for life on Earth...Capabilities:
- Answers factual questions
- Summarizes text
- Writes creative content (poems, stories, descriptions)
- Classifies text (sentiment, category)
- Generates lists and structured content
- Explains concepts in simple terms
Limitations:
- 512-token context window limits response length
- Trained on children's stories base -- may default to simple language
- Not as capable as larger instruction-tuned models
- English only
Training Pipeline
Instruction Dataset Sources
Quick Start
Option 1: Run directly
pip install torch tiktoken huggingface_hub
python nanogpt_slm_tinystories_instruct_inference.pyOption 2: Import and use ask() in your own code
from nanogpt_slm_tinystories_instruct_inference import ask
# Simple question
print(ask("What is the capital of France?"))
# With input context
print(ask(
instruction="Summarize the following text.",
input_text="Machine learning enables systems to learn from data..."
))
# Control generation
print(ask("Write a poem about the ocean.", temperature=1.0, top_k=100))Option 3: Load weights manually
from huggingface_hub import hf_hub_download
import torch
model_path = hf_hub_download(
repo_id="nishantup/nanogpt-slm-tinystories-instruct",
filename="nanogpt_slm_tinystories_instruct.pth"
)
from nanogpt_slm_tinystories_instruct_inference import GPT, GPTConfig
config = GPTConfig()
model = GPT(config)
model.load_state_dict(torch.load(model_path, map_location="cpu"))
model.eval()Prompt Format
The model uses the unified Task/Question/Answer format:
Task: {instruction}
Question:
{input} <- only if input is non-empty
Answer:
{response}Model Architecture
ask() API Reference
ask(instruction, input_text="", max_tokens=256, temperature=0.7, top_k=40)Files
Related Models (Vizuara SLM Family)
Citation
Eldan, R., & Li, Y. (2023). TinyStories: How Small Can Language Models Be
and Still Speak Coherent English? arXiv preprint arXiv:2305.07759.