Modelpublic

mj0530/CERA-MoA-Qwen3-4B-4agents

sourceHugging Faceapache-2.0updated 9d agoView on Hugging Face

2likes

Model Card

CERA-MoA (Qwen3-4B, 4 Agents)

Official checkpoint for CERA-MoA: Co-Evolving Routing Mechanisms with Continually Learning LLM Agents.

—Paper: arXiv:2609.18779
—Code: github.com/michaeljiang0530/CERA-MoA
—Hugging Face Papers: hf.co/papers/2609.18779

What's included

Path	Description
`agents/agent_{0,1,2,3}/`	Per-agent LoRA adapters (PEFT) on `Qwen/Qwen3-4B`
`router.pt`	Familiarity predictor / router heads

Backbone weights are not included; load Qwen/Qwen3-4B from Hugging Face and attach these LoRAs.

Quick load (agents)

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "Qwen/Qwen3-4B"
repo = "YOUR_HF_USERNAME/CERA-MoA-Qwen3-4B-4agents"

tokenizer = AutoTokenizer.from_pretrained(base, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="bfloat16", device_map="auto", trust_remote_code=True)

# Example: load agent 0
model = PeftModel.from_pretrained(model, f"{repo}/agents/agent_0")

For full multi-agent routing + inference, use the official code in the GitHub repo (train_gspo_multi_agent.py / evaluation scripts) and point it to this folder (or a local download of the Hub repo).

Checkpoint note

This is the final snapshot from the paper's main Qwen3-4B 4-agent run (midemb=True, marginal push-away), corresponding to training step 6000.