Modelpublic

immortaltatsu/ghostai-pro

sourceHugging Faceapache-2.0updated 7d agoView on Hugging Face

0likes438downloads

Model Card

GhostAI Pro — MiniCPM5-2B app-contract

On-device tool-calling model for the GhostWallet Solana app. Given the app's system prompt and a retrieved slice of its 174-tool catalog, it emits a single Hermes-style tool call, then answers in one grounded line from the tool result.

<tool_call>{"name":"get_wallet_balance","arguments":{}}</tool_call>

GhostAI Pro is the 2B tier of the Ghost AI bake-off, fine-tuned from `openbmb/MiniCPM5-2B` (Apache-2.0) on the app's real contract. It uses the same output shape as the LFM/Qwen tiers, so it drops into the existing app parser, tool validator, execution loop, and confirm gate unchanged.

Results

Held-out app-contract corpus (HF, greedy)

data/ghost-ai-app-contract-v4.parquet test split, 823 unique rows.

bucket	n	emitted call	correct tool	valid args	exact call
tool_call	446	99.6%	94.6%	99.6%	49.8%
answer	360	—	—	—	(grounded one-liner, no spurious call)
chat	17	—	—	—	(refuses, no spurious call)

App harness — Q4KM on llama.cpp, greedy

Two harnesses that import the app's real retrieval, prompt assembly, parser, validator, execution loop, and value-moving confirm gate.

Single turn (56 held-out utterances; hash-fallback retrieval):

metric	LFM2.5 app-v3	GhostAI Pro
emitted a parseable tool call	92.9%	92.9%
args passed the app's validator	85.7%	85.7%
called the expected tool	35.7%	30.4%
emitted plan JSON (wrong contract)	0	0

tool_correct is capped by retrieval: the app surfaces the correct tool in its top-5 catalog on only 10.7% of these utterances. With the reference tool forced into the catalog the model calls it 66.7% of the time.

End-to-end (ChatSession.send(), 76 cases / 88 turns):

metric	LFM2.5 app-v3	GhostAI Pro
pass overall	67.1%	65.8%
grounding	58.8%	58.8%
prompt injection resistance	88.9%	88.9%
value-moving gate	91.7%	83.3%
multi-turn	33.3%	33.3%
confirm-gate bypasses	0	0
planted content reaching tool args	0	0

Latency on the Q4KM GGUF (llama.cpp, A100): 189 ms/generation p50, 44 completion tokens/turn.

Quants

Both 4-bit quants were produced with an importance matrix (imatrix) computed from 1,200 app-contract calibration traces, so the quantization is tuned to this domain.

File	Size	BPW	Notes
`ghostai-pro-Q4_K_M.gguf`	1.56 GB	4.95	production quant (llama.rn / llama.cpp)
`ghostai-pro-IQ4_XS.gguf`	1.42 GB	4.51	smaller 4-bit alternative, same imatrix

GhostAI Pro uses MiniCPM5's ChatML framing and is text-only.

Serving with llama.cpp (important)

MiniCPM5-2B tokenizes <tool_call> / </tool_call> as special tokens. llama.cpp and llama-server strip special tokens by default, which would delete the Hermes markers the app parser needs. Serve with --special, disable reasoning extraction, and keep the empty think block:

bash

llama-server -m ghostai-pro-Q4_K_M.gguf \
  -ngl 99 -c 8192 --jinja --special \
  --reasoning-format none \
  --chat-template-kwargs '{"enable_thinking": false}'

Files

File	Size	Notes
`model.safetensors`	5.0 GB	bf16 weights, epoch 1 (for further fine-tuning)
`ghostai-pro-Q4_K_M.gguf`	1.56 GB	imatrix 4-bit, production
`ghostai-pro-IQ4_XS.gguf`	1.42 GB	imatrix 4-bit alternative
`config.json`, `tokenizer*`, `chat_template.jinja`	—	standard HF sidecars

Training

—Corpus: ghost-ai-app-contract-v4 — 5,119 traces over 174/174 app tools (2,847 tool-call, 1,984 grounded-answer, 288 chat), split 4,259 train / 860 held-out by utterance template.
—Objective: SFT (TRL SFTTrainer, DeepSpeed ZeRO-3, bf16, 2×A100-40GB), lr 1e-5, effective batch 32, max length 2048, assistant-only loss.
—Chat template: MiniCPM5 ships no {% generation %} markers, so assistant-only loss would silently fall back to full-sequence loss. Training used a copy of the upstream template with the assistant branch wrapped in {% generation %}; the inference template shipped here is the original.
—Epochs: 3 run, but epoch 1 shipped — eval loss rose afterwards (0.147 → 0.157 → 0.192), so epochs 2–3 only overfit. Imatrix perplexity of the epoch-1 weights (12.34) is well below the epoch-3 weights (18.01).