Pacific-i64/Token-Routing-Interactive-Paper

TR-HASH MoE 200M — Interactive Multi-Hash Report

Interactive companion for the 201.2M-parameter TR-HASH MoE full-SFT checkpoint selected from epoch 2 on PIQA.

The Space provides:

• —an architecture explorer for the shared SwiGLU and deterministic multi-hash top-2 residual path;
• —checkpoint-derived tokenizer IDs and expert assignments for selected tokens;
• —the recorded three-epoch full-parameter SFT and PIQA diagnostics;
• —live chat generation from AETHORIA-AI/TR-HASH-MoE-200M-160B-SFT.

The model has 16 layers, a 32k vocabulary, GQA 14/2, four stored residual experts, two independent token-ID hash routes per layer, and a shared dense SwiGLU branch. There is no learned expert router and no LoRA adapter.

Its metrics describe only this 201.2M model. They are training diagnostics and a full PIQA validation measurement, not a dense-baseline comparison or a broad general-capability benchmark.

Sources

• —TR-HASH MoE 200M full SFT
• —160B-source refinement checkpoint
• —Training framework
• —TR-Hash-i64