cloudsurf-software/CloudSurf-4B-FC

1---2# Hugging Face model card — CloudSurf-4B-FC (E1-rev champion, 2026-08-17).3# Name RULED 2026-08-17 (Brady): CloudSurf-4B-FC under HF org `cloudsurf-software`4# (namespace `cloudsurf` was already taken on HF by an unrelated account).5# "4B" = effective active params (the base's own E4B convention); total params6# ~8.0B are disclosed below.7license: apache-2.08language:9  - en10library_name: transformers11pipeline_tag: text-generation12base_model: google/gemma-4-E4B-it13base_model_revision: ee0ef6023621cff504d758262d4e04895a5af4a2  # HF main sha at release window (boxes pulled main 2026-08-16/17)14tags:15  - function-calling16  - tool-use17  - bfcl18  - cloudsurf19  - qlora20  - gemma-421datasets: []   # training mix is proprietary (fc-tier1); generators + gates documented in-repo22model_name: CloudSurf-4B-FC23inference: false24---25 26# CloudSurf-4B-FC — function-calling / tool-use model27 28> Gemma-4 E4B (effective-4B active; ~8.0B total params) QLoRA-tuned by29> CloudSurf Software for function calling and agentic tool use. **BFCL V430> FULL (harness pinned `f7cf735`): 3-seed mean 55.73 vs stock 34.81 (+20.9)**31> — above the published small-model class bar (Nanbeige4-3B 51.40) and above32> stock gpt-oss-20b (49.09) measured on the identical rig. Self-run numbers33> with full settings disclosure; gorilla PR open34> ([ShishirPatil/gorilla#1357](https://github.com/ShishirPatil/gorilla/pull/1357),35> official run invited).36 37## Scores (BFCL V4 FULL 22/22, prompt-mode, thinking ON, temp 0.001)38 39| | Overall | Non-Live | Live | Multi-Turn | Web | Memory | Irrelevance |40|---|---:|---:|---:|---:|---:|---:|---:|41| **CloudSurf-4B-FC (3-seed mean)** | **55.73** | 87.23 | 79.82 | 43.46 | 48.00 | 41.65 | 80.63 |42| Stock gemma-4-E4B-it (×3 mean) | 34.81 | 84.30 | 73.97 | 19.46 | 7.00 | 16.56 | 84.37 |43 44Raw result files for every run in this table — ours **and** the stock45baselines, plus per-category scores and a comparison chart — are public:46[CloudSurf-4B-FC-bfcl-results](https://huggingface.co/datasets/cloudsurf-software/CloudSurf-4B-FC-bfcl-results).47 48Seeds {42, 7, 11} → {55.53, 54.86, 56.81}, spread 1.95. Best-checkpoint49(seed 11) re-eval: two further same-box FULL evals returned 54.39 and 54.91 —50measured same-checkpoint eval band {56.81, 54.39, 54.91}, mean 55.37,51sd ≈ 1.05 (per-run BFCL FULL eval noise on this model class is ≈ ±1;52single runs < ~2.5 apart are not meaningfully different). Every measured53number, including the weakest (54.39), exceeds the published small-model54class bar (Nanbeige4-3B, 51.40) and stock by double digits.55Contamination-audited clean (0/255 eval-prompt 8-gram overlap; behavioral56trace verification).57 58**Serving-variant fairness pair** (pre-registered, measured 2026-08-17):59stripping stray `<tool_call|>` closers — a decode artifact of the base's60native prior — raises STOCK to 39.40 (Multi-Turn 19.46 → 33.12) and moves61this model +0.52 (→ 54.91). Under that matched best-serving frame the honest62comparison is **54.91 vs 39.40 (+15.5)**; the as-registered frame is 55.7363vs 34.81 (+20.9). Both are reported everywhere; the strip ships in the64proposed BFCL handler for all Gemma-4 prompt-mode models.65 66**Honesty disclosures** (full apparatus in the forthcoming paper):671. Web 48.00 is a *floor*: ~9% of web_no_snippet entries overflow the 131K68   context window mid-research (multi-hop page fetches) and score 0; the69   errors bias against this model (clean ceiling ≈ 50.5).702. Memory gain is trained env-API-namespace competence (schemas71   independently authored, namespace deliberately aligned) — legitimate but72   not zero-shot API transfer. Web *is* zero-shot (no web data in the mix).733. Format Sensitivity (excluded from Overall by the harness) regresses:74   FS max-delta ~45–81 vs stock ~10 — the tune is format-specialized to its75   registered serving template.764. A pre-registered serving-variant fairness pair (stray `<tool_call|>`77   closer strip, which also rescues stock's native decode quirk) was78   measured 2026-08-17 and is disclosed above (54.91 vs 39.40); headline79   numbers use the registered handler.80 81## Training82 83- Adapter: QLoRA nf4, LoRA r8 α16, all 7 linear projections, lr 5e-5,84  686 steps (~0.16 epochs of 34,926 per-turn examples), assistant-only loss.85- Data: 2,747-row fc-tier1 mix (`7f8711e00f66bad0`) — state-verified86  gpt-oss-120b teacher trajectories (partially-observable look-then-act envs),87  memory-discipline conversations, irrelevance-refusal rows; teacher reasoning88  retained in the thought channel.89- The decisive technique: **supervised span termination (render v2)** — one90  training example per assistant turn, history byte-exact to the serving91  handler, every supervised span terminated with a serve-time stop id92  (`<turn|>`). Full apparatus in the forthcoming paper.93 94## Repository contents95 96- `merged/` — BF16 merged champion checkpoint (seed 11).97- `adapters/seed-{42,7,11}/` — the three LoRA adapters (111M each) for98  repeatability; merge with `peft` or `scripts/fastloop_merge_adapter.py`.99- This card. Training data is not released; generators, gates, and the full100  measurement record are described in the paper (same posture as early101  xLAM/ToolACE releases).102 103## Release checklist (state at card freeze)104 1051. ✅ V-3 live board re-verify (2026-08-17 pull; snapshot archived).1062. ✅ V-4 FS-exclusion audit (Overall recomputed exactly from leaves).1073. ✅ V-6 web-poison mechanism (context overflow; biases against us).1084. ✅ V-5 name ruled: CloudSurf-4B-FC @ hf.co/cloudsurf-software.1095. ✅ V-1 best-checkpoint re-eval — pre-registered ±0.6 clause MISSED110   (54.39/54.91 vs 56.81); adjudicated as eval-noise mis-calibration111   (poison byte-identical across runs, Multi-Turn stable); measured band112   disclosed above in place of the clause.1136. ✅ V-2 fairness pair measured: stock 39.40 / champion 54.91 under the114   strip variant — both frames disclosed above.1157. ✅ Gorilla PR opened 2026-08-18:116   [ShishirPatil/gorilla#1357](https://github.com/ShishirPatil/gorilla/pull/1357)117   (handler ships the strip per V-2; raw results dataset linked above).118 119## Serving — OpenAI-compatible endpoint (τ²-bench setup)120 121The model's native interface is prompt-mode (Gemma-4 turn format, bracket122tool-call lists). For harnesses that speak the OpenAI chat/tools API —123including [tau2-bench](https://github.com/sierra-research/tau2-bench) — this124repo ships the exact serving bridge used for our τ²-bench leaderboard runs:125[`serving/fc_tau2_bridge.py`](./serving/fc_tau2_bridge.py) (stdlib-only, no126dependencies). Full τ²-bench trajectories (retail / airline / telecom, ×4127trials, unmodified) are public:128[tau2-trajectories-cloudsurf-4b-fc](https://huggingface.co/datasets/cloudsurf-software/tau2-trajectories-cloudsurf-4b-fc).129 130```bash131# 1) serve the merged weights with any /v1/completions backend, e.g. sglang:132python -m sglang.launch_server --model-path cloudsurf-software/CloudSurf-4B-FC \133  --revision main --port 30000   # serve the merged/ subtree134# 2) put the bridge in front (OpenAI-compatible /v1/chat/completions, tools in / tool_calls out):135python serving/fc_tau2_bridge.py --port 8000 --backend http://127.0.0.1:30000136# 3) point any OpenAI client at http://127.0.0.1:8000/v1 (model id: cloudsurf-4b-fc)137python serving/fc_tau2_bridge.py --selftest   # render/parse self-checks138```139 140Disclosure (mirrors our τ²-bench submission methodology): the bridge appends a141fixed function-calling instruction block to the caller's system message, and142when a generation terminates inside the thought channel with an empty answer it143attempts to recover a bracket-format tool-call list from the raw text before144returning an empty answer. Both behaviors are in the script, unabridged.145 146---147 148### 2026-08 weight-file fix149 150Earlier revisions of this repository shipped a `model.safetensors` missing 54151tensors relative to the base model: `k_proj`, `v_proj`, and `k_norm` weights152for language-model layers 24–41 (2076 keys instead of 2130). These layers use153shared KV projections (`num_kv_shared_layers: 18`), and the export path154(transformers 5.15.0 `save_pretrained`) silently omitted their parameters.155Some engines tolerated the omission; vLLM 0.25.1 correctly refused to load156the checkpoint.157 158The current revision restores all 2130 tensors. The 54 restored tensors are159byte-exact copies from `google/gemma-4-E4B-it` — this is provably lossless:160the fine-tuning adapter contains no LoRA entries for k/v projections at or161above layer 24, and every one of the 2076 retained tensors is either162bit-identical to the base or accounted for by an adapter-targeted module.163`config.json` now matches the base model's schema so the checkpoint loads164under both current and older transformers versions.165 166Verification: the fixed checkpoint serves under vLLM 0.25.1, and greedy167decoding logprobs match an independently reconstructed base+adapter merge at168the same numerical floor measured between two servings of the unmodified base169model (top-1 agreement 99.6% vs a 98.8% same-model cross-engine control; max170chosen-token logprob difference 0.166 vs 0.126 control).171