Modelpublic

Zhongzhu/tunekv-terminalbench-27b

sourceHugging Faceapache-2.0updated 18h agoView on Hugging Face

0likes

Model Card

TuneKV · Terminal-Bench · Qwen3.8-27B

Trained prefix K/V (the "differentiable harness": frozen weights, trained KV cache at a fixed prompt prefix) for Qwen/Qwen3.8-27B (snapshot 1d4bf0f2) on Terminal-Bench, with same-stack base replicates.

Protocol

—Fixed 80-task set, terminus-2 agent, driver run_eval_fixed.py (sha 953b0d4d) + frozen adapter run_native.py (sha 8cf915c5), terminal-bench 0.2.18; temperature 0.7, max_tokens 4096, context 131072, 2 concurrent trials; tasks without a verdict count as 0.
—Serving: vLLM 0.28 (image tunekv:multi-swe-0922), one B300 per server (TP1), FLASHINFER attention, prefix caching off, limit-mm 0, gdn_prefill_backend=triton; default chat template (thinking on, effort xhigh). Tuned servers add the tunekv_vllm.hybrid connector (n_inject 704); base and tuned share every other flag.
—Prefix: 722-token shared head of the harness prompt under the 27B tokenizer; 704 trainable positions (64-aligned GDN boundary): 16 full-attention layers' K/V + 48 GatedDeltaNet boundary states (state1 trained, conv state0 re-captured coherently at export).

Results (resolved / 80)

arm	runs	mean	Δ vs base	per-task sign test (tuned vs 8 base runs)
base (same stack)	b1 42, t1 47, t2 38, t3 47, b2 46, b3 41; concurrent controls b4 43, b5 43	43.5 (6 canonical, SD 3.7)	—	—
CE + 0.1·forward KL	40, 43	41.5	−2.0 (b1: −2, McNemar p 0.51)	14 up / 17 down, p 0.72
GRPO (clip 0.2 + 0.1·reverse KL, CE 0)	38, 36	37.0	−6.5 (b1: −4 / −6, p 0.55 / 0.15)	9 up / 21 down, p 0.043

CE is null; GRPO regresses (both runs at or below the lowest base run). eval/COMPARISON_27B_{CE,GRPO}.json and eval/SUMMARY_TB6.json hold per-task outcomes, pairings and tests.

Arms

—CE + 0.1 KL (ce/): 27B's own successful training-pool trajectories (12 tasks outside the eval set, 4 trials each, 26 successful trials) → V3-style clean rows (canonical JSON answer, observation-error filter, repeat guard) = 148 rows; capture init, lr 1.5e-3 cosine, 3 epochs = 105 steps, window 32768.
—GRPO (grpo/): 4 on-policy base trials per eval task at temperature 0.7 (b1 + t1..t3), reward = trial is_resolved, per-task z-score; every LLM call from the journal = one row (4005 rows ≤ 32768 tokens, 19 mixed tasks); lr 3e-4 cosine, 624 steps, effective batch 2.

Layout

—ce/, grpo/: serving artifact (export/artifact: 64 files + COMPOSITION.json + READY), row reports, training config, per-step training log, injection canary (GATE*.json); grpo/eval/ = the GRPO tuned + base verdicts used in the comparison.
—eval/<run>/: lean verdicts (launch intent, completion receipt, per-trial results.json) for every run.
—trajectories/<run>.tar.gz: full run directories (HTTP journal of every model call, agent logs, tmux sessions).
—train/*.tar.gz: episodes, training rows, training/export logs and checkpoints (dataset caches excluded).
—code/: pipeline scripts (serve, eval, row builders, config writers, export, canary, comparison).

Serving the artifact

bash

KVC='{"kv_connector":"TuneKVConnector","kv_role":"kv_both","kv_connector_module_path":"tunekv_vllm.hybrid",
      "kv_connector_extra_config":{"prefix_json":"<artifact>/prefix_ids.json","kv_dir":"<artifact>","n_inject":704}}'
vllm serve Qwen/Qwen3.8-27B --tensor-parallel-size 1 --max-model-len 131072 --no-enable-prefix-caching \
  --attention-backend FLASHINFER --limit-mm-per-prompt '{"image":0,"video":0}' \
  --additional-config '{"gdn_prefill_backend":"triton"}' --kv-transfer-config "$KVC"

Requests only receive the trained KV when their rendered prompt starts with the 704 prefix ids. W&B: zhoumllab/TERMINALBENCH runs 5z1z9t5g (CE), e0ikai7l (GRPO), oufrbh9e (eval summary).