Modelpublic

hyperspaceai/thor-1-lora

sourceHugging Faceapache-2.0updated 7mo agoView on Hugging Face

1likes8downloads

Model Card

THOR-1: Unified MoE Brain

THOR-1 is a LoRA adapter for Qwen3.5-35B-A3B (MoE: 35B total params, 3B active) that unifies 6 AI orchestration capabilities into a single model.

Capabilities (6 modes)

Mode	Tag	Purpose
Intent	`[INTENT]`	Classify user intent and extract parameters
Decompose	`[DECOMPOSE]`	Break complex tasks into executable DAG steps
Execute	`[EXECUTE]`	Iterative REPL-style code execution
Predict	`[PREDICT]`	Predict execution outcomes (world model)
Route	`[ROUTE]`	Route inference across providers/models
Rank	`[RANK]`	Rank and select agents/skills/tools

Each training example has a mode tag prefix in the system prompt that teaches the model to activate different expert mixtures per task type.

Training Details

—Base model: Qwen3.5-35B-A3B (hybrid linear attention + Mamba SSM + 256 MoE experts)
—Method: LoRA (r=16, alpha=32, targets: qproj + vproj)
—Training data: 6,384 examples across 6 modes (SFT format)
—Epochs: 3 (1,077 steps)
—Best validation loss: 0.2504
—Training time: 12.9 hours on NVIDIA H100 80GB
—Precision: BF16 with BitsAndBytes 4-bit quantization (NF4)
—Optimizer: AdamW 8-bit, LR=2e-5 with cosine decay

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-35B-A3B", trust_remote_code=True)
model = PeftModel.from_pretrained(base_model, "hyperspaceai/thor-1-lora")
tokenizer = AutoTokenizer.from_pretrained("hyperspaceai/thor-1-lora")

prompt = "<|im_start|>system\n[INTENT] Classify the user's intent.\n<|im_end|>\n<|im_start|>user\nBook a flight to Tokyo next Friday<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Part of the Thor Ecosystem

THOR-1 is the unified brain of the Thor AI orchestration system, powering intelligent task routing, agent selection, and execution planning across 21 named agents and 1,200+ community agents.

Developed by: HyperspaceAI