cow9000/aicivs-joint-p1-v3b2-gguf

aicivs-joint-p1-v3b2-gguf

The AICivs student: MiniCPM5-2B fine-tuned (LoRA r64, merged) to power the NPCs of AICivs, a Minecraft mod where villages are living civilizations — villagers with personalities, memories and relationships, quests grounded in the world around the giver, wars, famines, successions, gossip and authored histories. One 2.5B-parameter model answers every operation of the mod's service contract, so the whole runtime fits in about 8 GB of GPU memory next to the game.

This build: model-q4_k_m.gguf, llama.cpp Q4KM of the merged weights (~1.6 GB), for the structured/authoring tiers where llama.cpp's grammar is needed (it enforces the contract's regex patterns). Quests on this build: 16/16 schema, 15/16 validator, 14/16 clean, p50 2.1 s on an RTX 3090 Ti with four 8k slots (-c 32768 -np 4).

What it is for

The model answers nine operations, each a fixed JSON schema (AICivs contract 1.0, prompt version 1):

It is not a general chat model. It expects the service's rendered prompts (a short system prompt naming the speaker, a user turn carrying the context and the allowed actions) and answers in the contract's JSON. Outside that framing it will still answer, but nothing about the game's grounding rules applies.

How to use it

The intended path is the AICivs stack: ops/aicivs-up.sh in the mod's repository downloads this repo, starts the servers and the service, and the mod calls the service. To serve the model yourself:

docker run --gpus all -p 8001:8080 -v $PWD/aicivs-joint-p1-v3b2-gguf:/models/student:ro ghcr.io/ggml-org/llama.cpp:server-cuda-b10236 \
  --host 0.0.0.0 --port 8080 --alias aicivs-joint-p1-v3b2 -m /models/student/model-q4_k_m.gguf -c 32768 -np 4 -ngl 99 -fa on

Four 8k slots (quest and history prompts run up to 5.5k tokens). ~3 GB of GPU. Use this build for the structured and authoring operations; the W4A16 build on vLLM is the faster choice for dialogue and gossip.

Send OpenAI-style chat completions with response_format: {"type": "json_schema", ...} carrying the operation's schema (the pydantic models in service/aicivs_service/contract/) and chat_template_kwargs: {"enable_thinking": false} — the model was trained with thinking off. Temperature 0.8 for dialogue and gossip, 0.6 for the structured operations.

A real dialogue.respond exchange, exactly as the service renders it:

system: You are Kagard Mikae, child of Nolhou. Voice: childlike — e.g. "I'm Kagard Mikae! Are you from far away?" /
        "I'm not supposed to talk to strangers.".
        Traits: timid. Mood: neutral. Toward the speaker you are devoted to; quirk: counts on fingers.
        Answer in character, at most 30 words, as JSON. Never mention a person, place, item or event that is not
        listed below; use listed ids exactly. Only use actions you are told you may use.
user:   You may: end the talk (end_conversation) / ask for: tag:minecraft:iron_ingot, tag:minecraft:emerald /
        reveal: topic:nolhou_h1 / show emotion (express_emotion).
        Conversation:
        Player: What a miserable little life you have.

{"speech":"I'm not supposed to talk to strangers. Please don't be mean to me.","player_intent":"insult",
 "sincerity":"honest","actions":[],"relationship_delta":{"trust":-10,"affection":-15,"respect":-5,"fear":0},
 "patience_delta":-0.2,"mood_after":"afraid",
 "memory_write":{"summary":"Player insulted my life and me.","salience":0.8,"valence":-1.0,"tags":["insult","fear","hurt"]}}

Two things the serving side must do, because grammar backends do not: clamp numeric bounds (vLLM's grammar ignores minimum/maximum, so a -1.1 valence must be clamped, not rejected) and validate references — every id in an answer is checked against the request's manifest, and the game repairs or falls back on a miss.

How it was made

Prompt distillation (the mod's docs/08-finetuning.md): a 12-civilization world simulated headless for 120 seasons supplied souls, memories, chronicles and manifests; Qwen3.8-27B answered long, rule-heavy prompts (with a teacher-played player for multi-turn conversations, including nonsense lines because players are nonsensical); the answers were filtered through the service's validator plus the mod's judgment layers (feasibility, authority, faction, economics, narrative) and became targets under the service's short prompts. ~35k rows across the nine operations (22k dialogue turns, 2.7k quests, 1.1k repairs, 2.4k personality batches, 2.0k consolidations, 2.3k narrations, 0.5k histories, 0.6k summaries, 3.5k gossip lines), LoRA r64 on MiniCPM5-2B, three rounds (dialogue, then joint, then a 6k-token-window pass because quest prompts run 4–5.5k tokens).

Guidance-off evaluation on held-out rows (JSON must be right without a grammar): dialogue validator 99.9 %, quest.generate 95–98 % validator / 94–97 % fully clean, quest.repair 94 %, personality 99 %, consolidate 98.5 %, narrate 97 %, history 100 %, summarize 100 %, gossip 100 %. Against the 27B teacher's own dialogue answers a teacher-as-judge preferred the student 30 %, tied 24 %.

Limitations

• —Prompt version 1 only; a new contract version needs a re-distilled model (the pipeline is in the repository).
• —Knows the vocabulary it was trained on: the primitives, actions, intents, registers and event types of contract 1.0. New items, landmarks, civs and names are fine (it copies ids from the request); a new primitive or register is not.
• —English only. Writing quality is below the 27B teacher's; structure and grounding are the point.

License

Apache 2.0, as the base model. Trained data was synthetic (a simulated world and a 27B teacher).