Jongbin-kr/llama-3.1-8b-instruct-4x1-moe

Llama 3.1 8B Instruct 4x1 MoE

This checkpoint sparse-upcycles every Llama 3.1 8B Instruct decoder MLP into four initially identical experts with dropless top-1 routing.

Load it with AutoModelForCausalLM.from_pretrained(..., trust_remote_code=True).