vkqwert12/tars-v1-medical

TARS v1.0 - Medical SFT

TARS is an experimental Large Language Model featuring a hybrid Parallel Mamba-3 and Fractal MoE architecture.

Key Architectural Specs

• —Sequence Mixer: Parallel Mamba-3 (Trapezoidal Discretization)
• —Expert System: Hierarchical Fractal MoE (4 Domain Experts, 2 Task Experts per Domain)
• —Memory: Continuous L3 Memory Bank
• —Latent Blocks: Quantum-Phase Routers and HDC Projection Layers

Training History

1. 1.Pre-training: 100,000 steps on FineWeb-Edu (10BT Sample).
2. 2.SFT: 5,000 steps on MedQuAD Q&A pairs.
3. 3.Alignment: 800 steps of Diversity-Enhanced Alignment.

Technical Details

• —Vocab Size: 32000
• —D_Model: 256
• —Layers: 4