KikoCis/FastContext-1.0-4B-longctx-imatrix-GGUF

[image]

<div style="border:2px solid currentColor; font-family:ui-monospace,'SF Mono','Cascadia Mono',Consolas,'Liberation Mono',monospace;"> <div style="border-bottom:1px solid currentColor; padding:6px 12px; font-size:11px; letter-spacing:3px; text-transform:uppercase; opacity:0.7; text-align:center;">KIKOCIS // LONG-CONTEXT IMATRIX GGUF // PRESERVED &amp; QUANTIZED</div> <div style="padding:14px; display:flex; flex-wrap:wrap; align-items:center; justify-content:center; gap:18px;"> <pre style="margin:0; flex:0 0 auto; font-family:ui-monospace,'SF Mono','Cascadia Mono',Consolas,monospace; font-size:9px; line-height:1.15; letter-spacing:0;"> repo/ ┌─────────┐ ├── src/ ═══════▶│ ◉ 4B │ │ ├── auth.rs ◀═══════│ scout │ │ └── db.rs └─────────┘ ├── lib/ READ·GLOB·GREP │ └── core.rs ──▶ auth.rs:41-77 └── tests/ ──▶ core.rs:102-130 256K ctx only what you need </pre> <div style="flex:0 1 auto; max-width:100%; text-align:center;"> <div style="font-size:23px; font-weight:800; letter-spacing:1px;">FASTCONTEXT-1.0-4B</div> <div style="font-size:12.5px; letter-spacing:1px; opacity:0.8; margin-top:5px;"><span style="white-space:nowrap;">256K REPO-EXPLORER</span> · <span style="white-space:nowrap;">QWEN3 DENSE 4B</span> · <span style="white-space:nowrap;">LONG-CTX IMATRIX</span> · <span style="white-space:nowrap;">8.0 GB → 1.96 GB</span></div> </div> </div> <table style="display:table; table-layout:fixed; width:100%; margin:0; border-collapse:collapse; font-family:ui-monospace,'SF Mono',Consolas,monospace; font-size:12px;"> <tr> <td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">FORMAT</div><div style="font-weight:700;">GGUF · IQ3M / Q4K_M</div></td> <td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">SIZE</div><div style="font-weight:700;">1.96 / 2.50 GB</div></td> <td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">ARCH</div><div style="font-weight:700;">QWEN3 DENSE · 36L</div></td> <td style="border-top:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">CONTEXT</div><div style="font-weight:700;">256K NATIVE</div></td> </tr> <tr> <td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">IMATRIX</div><div style="font-weight:700;">LONG-CONTEXT CALIB</div></td> <td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">RETRIEVAL @5K</div><div style="font-weight:700;">30/30 = BF16</div></td> <td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">RUNS ON</div><div style="font-weight:700;">METAL·CUDA·CPU·VULKAN</div></td> <td style="border-top:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">LICENSE</div><div style="font-weight:700;">MIT</div></td> </tr> </table> </div>

*Microsoft open-sourced it, then deleted it from HuggingFace and GitHub (verified: 404 on both). These are long-context-imatrix GGUF quants so the weights stay in your hands — the full preserved original (bf16, 8.0 GB) is at [KikoCis/FastContext-1.0-4B-SFT](https://huggingface.co/KikoCis/FastContext-1.0-4B-SFT). Own your AI.*

🔍 What FastContext is (and why it's special)

FastContext isn't a chatbot — it's a repository-exploration subagent for coding agents. Your main agent (Claude Code, Copilot, Cursor, OpenHands…) delegates file discovery to it:

1. 1.Main agent asks: "where is auth handled?"
2. 2.FastContext fires parallel read-only tool calls — READ / GLOB / GREP — across the repo,
3. 3.and returns just the file paths + line ranges you need as compact, focused context.

Your expensive frontier agent stops burning tokens crawling directories. Microsoft's (now-deleted) announcement reported ~60% fewer tokens from the main agent and +5.5% on SWE-bench — their figures, not independently reproduced here.

📦 Which file should I pick?

_{K-quants (Q4_K_M) = solid general quants. I-quants (IQ3_M) = smaller at similar quality; they need an imatrix (we ship ours: fastcontext4b.imatrix).}

What's different vs the other FastContext GGUFs: the importance matrix here is calibrated on long, multi-thousand-token sequences (LongAlign), not the usual short generic corpus — matching the 256K regime this model was built for. For AMD Strix Halo specifically, see plunderstruck's ROCmFP4 build (different target, code-weighted imatrix).

🧮 Will it fit? (RAM/VRAM cheat-sheet)

Total ≈ weights + KV-cache (KV grows with context):

🚀 How to run it

# llama.cpp — point it at your repo dump, ask for locations:
llama-cli -m fastcontext4b.Q4_K_M.imx.gguf -c 32768 \
  -p "…repo contents…\n\nWhere is authentication handled? Return file:line ranges only."

# llama-server (use it as a subagent endpoint for your main coding agent):
llama-server -m fastcontext4b.Q4_K_M.imx.gguf -c 65536 --port 8091

# Ollama (Modelfile included, 32K default):
ollama create fastcontext -f Modelfile && ollama run fastcontext

Recommended sampling: temperature 0.6, topp 0.9, topk 20. For pure retrieval calls, temperature 0 works well. Subagent pattern: keep FastContext resident on a cheap local endpoint; have your main agent call it for "where is X?" queries and inject only the returned ranges into its own context.

📊 Validation — measured on these files (honest)

Needle-in-haystack retrieval (find an inserted fact inside real long documents), greedy decoding:

At 5K context all three — including the aggressive IQ3_M — match the original bf16 perfectly: quantization is lossless for retrieval here. Deeper long-context numbers will be added once measured on a clean harness — no placeholder claims.

• —Harness: llama-server + OpenAI-compat API, temp 0, 30 tasks, haystacks built from real LongAlign documents, deterministic gold.
• —Date: 2026-07-02.

⚠️ Good to know

• —Strengths: repo exploration, long-document retrieval, read-only tool calling (READ/GLOB/GREP), returning compact file:line evidence.
• —It's a scout, not a solver — pair it with your main coding agent; don't expect it to write the patch itself.
• —The original repo is gone, so upstream docs/issues are gone with it; the harness conventions above are from the model's own announcement and community usage.

🗒️ Changelog

• —2026-07-02 v1 — IQ3M + Q4K_M with long-context imatrix; retrieval validated @5K (30/30 all); imatrix + Modelfile included; original preserved in the sibling repo.

📚 Credit & license

Model, weights, training: © Microsoft — FastContext-1.0-4B-SFT (MIT), sourced via the ShaunGves re-upload after the original was removed. Quantization + long-context imatrix + validation: KikoCis. MIT (same as upstream). No weights modified — faithful quantization only.