CoolFace
Modelpublic

SmallScale/Simple-Stories-Hindi-10M

sourceHugging Facemitupdated 2mo agoView on Hugging Face
1likes37downloads
Model Card

📖 Simple-Stories-Hindi-10M (11.45M Parameters)

A 11.45M parameter decoder-only Transformer language model trained from scratch on 2.11 million Hindi simple stories. The model generates coherent, creative, and grammatically sound Hindi stories given a short text prompt.

📊 Evaluation & Training Metrics

Metric / PropertyValue
Best Validation Loss`1.8157` (Cross-Entropy Loss)
Total Training Steps202,000 steps
Total Parameters11,453,120 (11.45M)
Non-Embedding Parameters10,173,120 (10.17M)
Training DatasetSmallScale/Simple-Stories-Hindi (~2.11M stories)
Model Size on Disk~45.8 MB (model.safetensors)

🏗️ Model Architecture Details

ParameterValueNotes
ArchitectureLLaMA-style DecoderRoPE + SwiGLU + RMSNorm
Hidden Size (`d_model`)320Vector dimension
FFN Intermediate Size8968/3 × d_model rounded to multiple of 64
Layers (`n_layers`)7Transformer blocks
Attention Heads (`n_heads`)5Multi-Head Self Attention
Head Dimension64d_model / n_heads
Context Length (`max_seq_len`)512 tokensSequence window
Vocabulary Size4,000SentencePiece Unigram (Devanagari optimized)
Weight TyingEnabledToken embeddings & output projection share weights
Precisionfloat32Weights stored in native FP32 safetensors

🚀 Quick Start & Usage

python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load tokenizer and model directly from Hugging Face
tokenizer = AutoTokenizer.from_pretrained("SmallScale/Simple-Stories-Hindi-10M", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("SmallScale/Simple-Stories-Hindi-10M", trust_remote_code=True)

if torch.cuda.is_available():
    model = model.to("cuda")

# Prompt input
prompt = "एक समय की बात है"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

# Generate story
outputs = model.generate(
    **inputs,
    max_new_tokens=200,
    do_sample=True,
    top_k=40,
    top_p=0.95,
    temperature=0.8
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

📝 Sample Generated Stories

Prompt: एक समय की बात है

एक समय की बात है, और मैं छाया से देखता हूं। मेरे दो लोग, जीन और सैमुअल हैं, जो एक भव्य यात्रा पर जा रहे हैं। वे एक ही स्थान पर रहते हैं, लेकिन वे दोनों अपनी-अपनी कहानियाँ चाहते हैं...

🔗 Related Resources

📄 License

MIT License