Drissman/hermythos-rdt

HERMYTHOS — Agentic AI Engine + Ternary RDT Model

HERMES (Rust agentic engine) + MYTHOS (1.58-bit ternary RDT model). One sovereign binary. Zero Big Tech dependence.

<p align="center"> <strong>Qwen3-8B base → Ternary Bonsai weights → QAT fine-tuned → Q4KM quantized</strong><br> <em>4.68 GB · 4.90 BPW · 399 layers · 73.5% QAT accuracy · CPU-native</em> </p>

Why HERMYTHOS exists

Every frontier model runs on someone else's cloud. Every agent platform phones home. Every fine-tune assumes NVIDIA.

HERMYTHOS breaks all three assumptions.

• —Sovereign inference — runs CPU-only on a laptop (5 tok/s on Intel Core Ultra, Q4KM). No GPU required. No API key needed.
• —Open weights — 1.58-bit ternary model derived from Qwen3-8B via QAT on H100, quantized to Q4KM (4.7 GB). You own every parameter.
• —Agent-native engine — 22 Rust crates (29 total), cybernetic loop with 8 tools, 3 frontends (TUI / Flutter Web / Open WebUI). Zero Python at runtime.

The model itself is only half the equation — the Rust engine (hermythos-server) provides the agentic scaffolding: tool execution, FSM-based cybernetic loops, memory persistence, and multi-agent LLM debate via RecursiveMAS.

Quick Install

# One command. Downloads the model + engine.
curl -fsSL https://raw.githubusercontent.com/drissman/hermythos/main/scripts/install.sh | bash

Or manually:

# 1. Get the model
hf download Drissman/hermythos-rdt bonsai-rdt-q4_k_m.gguf --local-dir ./models

# 2. Clone the engine
git clone https://github.com/drissman/hermythos
cd hermythos-rdt
cargo run -p hermythos-server --release -- --model ./models/bonsai-rdt-q4_k_m.gguf

Technical Specs

Why Ternary Matters

Standard LLMs use 16-bit floats per weight. That's 16 GB for an 8B model. Ternary packs weights into {-1, 0, +1} — 12-16× denser — and eliminates multiplication from inference entirely.

FP16 matmul:  multiply-add-multiply-add...  (expensive)
Ternary matmul: add-skip-subtract...        (just additions)

This means:

• —Runs on CPU — no GPU required. Laptop-grade Core Ultra gets 5 tok/s.
• —Runs on RISC-V — no CUDA dependency. No NVIDIA lock-in.
• —12× smaller — 4.7 GB fits in RAM + disk of any machine built after 2015.

The trade-off is training complexity: ternary quantization requires QAT (Quantization-Aware Training) with Straight-Through Estimator. This model was fine-tuned on an H100 — but inference runs anywhere.

Architecture — 3-Layer Stack

┌─────────────────────────────────────────┐
│  UI LAYER — 3 Frontends                  │
│  TUI (ratatui) · Flutter Web · Open WebUI│
└──────────────┬──────────────────────────┘
               │ WebSocket / OpenAI API
┌──────────────▼──────────────────────────┐
│  ORCHESTRATION — Rust/Tokio (22 crates)  │
│  Agent Core · Tools · Memory · Skills    │
│  RecursiveMAS · MDASH · Cluster · Faber  │
└──────────────┬──────────────────────────┘
               │ GGUF · llama.cpp
┌──────────────▼──────────────────────────┐
│  MODEL — BonsaiRDT Ternary               │
│  Qwen3-8B base · BitLinear · 252 layers  │
│  1.58-bit weights · 4.68 GB Q4_K_M       │
└─────────────────────────────────────────┘

The engine (22 crates):

• —hermythos-server — WebSocket + OpenAI-compatible API backend
• —hermythos-mas — RecursiveMAS multi-agent topologies
• —hermythos-cluster — Distributed GRPO + debate orchestration
• —hermythos-compute — TERNARY format, CPU backend (AVX2)
• —faber-* — Sovereign data platform (Faber Foundry, MIT)

270+ tests. `cargo test --workspace` = 100% green.

Performance

Rule: CPU-only on WSL2. GPU path is slower due to D3D12 overhead.

Roadmap — 100 Days

✅ Distribution  →  ⬜ Documentation  →  ⬜ External Testing  →  ⬜ Auto-Evolution

Persona-Driven Design

HERMYTHOS ships with 6 interaction modes tuned by BMAD (Brain-inspired Multi-Agent Distillation):

Credits

• —Architecture & Training: Driss NAAMANE (Senior Cloud Architect / TDA)
• —Base Model: Qwen3-8B (Alibaba) + Ternary Bonsai (prism-ml)
• —QAT Pipeline: H100 RunPod, TRL + PEFT
• —Quantization: llama.cpp Q4KM
• —Engine: 22 crates Rust, 270+ tests, Apache 2.0 / MIT dual-licensed

License

Model weights: Apache 2.0 (inherited from Qwen3) Engine: MIT

"Un modèle que tu possèdes. Un moteur que tu contrôles. Une plateforme que personne ne peut t'enlever."