CoolFace
Modelpublic

build-small-hackathon/mind-of-tashi-micro-sft

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face
0likes42downloads
Model Card

The Mind of Tashi — micro student (SFT)

The opponent's mind for The Mind of Tashi, a simultaneous-commit ritual fighting game where the model's <think> block is the game (surfaced to the player as the "mind-scroll"). This is the transformers checkpoint; the GGUF the playable Space loads via llama.cpp lives at `build-small-hackathon/mind-of-tashi-micro-sft-gguf`.

David vs Goliath. The student is a custom MoE — loggenix-moe-0.4B-0.2A (~0.4B total / ~200M active per token) — SFT'd to read an opponent and commit blind, in an English + Hindi/Sanskrit (IAST) code-switched register. It is 10–100× smaller (active) than the API teachers it learned from.

Output contract

Emits a <think>…</think> block then one JSON line {"move": …, "taunt": …}. The <think> is product (rendered to the player), not debug. The host parses defensively and falls back to a legal move if generation is malformed.

Training

  • —Method: TRL SFTTrainer, completion-only loss (masked to the assistant turn), Modal L4, bf16, seq 4096, 3 epochs, bs=1 / grad_accum=4, warmup 10%, LR 2e-4.
  • —Data: `build-small-hackathon/mind-of-tashi-selfplay`, configs sft + sft_multiturn — self-play traces vs a frontier-API teacher pool, plus real-player matches.
  • —Recipe: TRL SFTTrainer on Modal L4 — the full hparams are above (training scripts are run off-Space and kept private).

⚠️ norm_topk_prob — required for llama.cpp

The base ships norm_topk_prob=false (raw top-k expert routing), but llama.cpp's qwen3moe graph hardcodes `norm_w=true` and ignores the GGUF expert_weights_norm key. A checkpoint trained with false produces garbage on every llama.cpp runtime. This model is trained with `norm_topk_prob=true` so the weights match llama.cpp's renormalised routing — that is what makes the GGUF coherent.

Eval

  • —Format gate (<think> + parseable {move,taunt} + legal move): 20/20 via transformers; via llama.cpp on the GGUF, f16 18/20, Q4KM 20/20, ~19–20/20 bilingual across 5 personas.
  • —Ladder gauntlet (mirror match across the 10-persona ladder vs tier-matched teachers): baseline 80/100 (8W/2L).

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("build-small-hackathon/mind-of-tashi-micro-sft")
model = AutoModelForCausalLM.from_pretrained("build-small-hackathon/mind-of-tashi-micro-sft")
# messages = [{"role":"system","content": <persona prompt>},
#             {"role":"user","content": <arena state + history>}]

Part of the bundle

Game Space · self-play dataset · SFT model (this) + GGUF · OpenEnv gym · GRPO model + GGUF — all under build-small-hackathon/mind-of-tashi-*.