Mostafa8Mehrabi/qwen3-50m-fp32

Qwen3-50M with GPT-2 Tokenizer

A 50M parameter version of Qwen3-0.6B using GPT-2's tokenizer for better compatibility.

Model Details

• —Base Model: Qwen/Qwen3-0.6B
• —Architecture: Qwen3 (8 layers, 384 hidden size)
• —Parameters: ~50M (reduced from 637M)
• —Tokenizer: GPT-2 (50,257 vocabulary)
• —Vocabulary: Reduced from 151,936 to 50,257 tokens

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

# Standard HuggingFace usage - no special flags needed
tokenizer = AutoTokenizer.from_pretrained("Mostafa8Mehrabi/qwen3-50m")
model = AutoModelForCausalLM.from_pretrained("Mostafa8Mehrabi/qwen3-50m")

inputs = tokenizer("Hello, how are you?", return_tensors="pt")
outputs = model.generate(**inputs, max_length=50, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Key Features

• —✅ Standard GPT-2 tokenizer (no trustremotecode)
• —✅ Compatible vocabulary sizes
• —✅ Works like any HuggingFace model
• —✅ 13x smaller than original Qwen3-0.6B

Architecture Comparison