witcheer/local-agentic-coding-bench-8gb-vram-2026-05

agentic coding benchmark: local LLMs on 8GB VRAM

can local LLMs do agentic coding (multi-turn tool calling, file creation, debugging) on consumer hardware? this dataset captures real test results.

hardware

• —GPU: NVIDIA RTX 4060 Ti 8GB
• —CPU: Intel i7-14700F
• —RAM: 32 GB DDR5
• —OS: Windows 11 + WSL2 (Ubuntu)
• —inference: llama-server (turboquant fork of llama.cpp)

what was tested

two agent frameworks:

• —Hermes Agent (NousResearch): structured tool calling with patch/write/bash tools
• —Pi (badlogic): simpler 4-tool interface (read/write/edit/bash)

two coding tasks at different difficulty levels:

• —portscout (easy): write a single-file concurrent port scanner using stdlib. create, test, done.
• —logpulse (hard): write a multi-file CLI log watcher with real-time stats, regex alerts, a fake log generator, a README, and self-test. requires file creation, debugging, and iterative fixes.

five model configurations:

• —Qwopus3.5-9B-Coder (9B dense, 43 tok/s, fully in VRAM)
• —Qwen3.6-35B-A3B MoE + thinking (35B MoE, 3B active, 35 tok/s, ncmoe=30, reasoning on)
• —Qwen3.6-35B-A3B MoE no-think (35B MoE, 3B active, 35 tok/s, ncmoe=30, reasoning off)
• —gpt-oss-20b (21B MoE, 3.6B active, ncmoe=30, OpenAI Apache 2.0, native MXFP4 FFN)
• —OmniCoder-9B (9B dense, 42 tok/s, Tesslate, based on Qwen3.5-9B, fine-tuned on 425K agentic trajectories)

key findings

1. 1.9B models break on complex tool calls regardless of framework. Qwopus 9B generated good code but failed on structured output: malformed patch JSON in Hermes, infinite reasoning loops in Pi. the model writes code fine but can't plan multi-turn agent actions reliably at 9B params.

1. 1.35B MoE is reliable but too slow for agent loops. Qwen3.6-35B-A3B never broke a tool call, but expert offload adds 2-5s latency per API call. agent tasks need 20-50+ round trips, compounding to 18-87 minutes for tasks cloud APIs finish in seconds.

1. 1.thinking mode is unusable for agentic work. reasoning tokens consumed most of the generation budget, inflating a 18-minute task to 49 minutes (2.7x overhead).

1. 1.context window is a hard constraint. the 35B model hit the 32K context limit during a debug loop on the hard task. bumping to 40K helped but caused OOM at 48K+ on 15 GB WSL memory.

1. 1.code quality was good across all configs. every model produced clean, working code. the bottleneck is agent loop speed and tool call reliability, not code generation quality.

1. 1.gpt-oss-20b is the breakthrough. OpenAI's 21B MoE (3.6B active) completed both easy and hard tasks quickly via Pi. only 1.8 GB VRAM with ncmoe=30, used just 26% of context on the hard task. first model to complete both agentic tasks on 8GB VRAM. the combination of OpenAI's structured output training + low VRAM footprint + Pi's simple tool interface is the winning formula.

1. 1.agent framework matters as much as the model. Pi's 4-tool interface (read/write/edit/bash) produces fewer tokens per turn and simpler tool call schemas than Hermes Agent's patch-based approach. this reduces context pressure and makes structured output easier for smaller models.

1. 1.agentic fine-tuning helps code quality, not agent loops. OmniCoder-9B (Tesslate, 425K agentic trajectories on Qwen3.5-9B) produced the cleanest first-shot code of any model tested: 83-line portscout in <1 min with no self-fixes needed. but it failed on the hard task the same way Qwopus 9B did: ran a blocking command without timeout, then got stuck in a generation loop. code generation and agent loop management are separate skills, and fine-tuning on coding trajectories doesn't fix the latter.

1. 1.thinking mode is a trap for 9B models. OmniCoder-9B with default thinking mode burned all tokens on reasoning_content with 0 code output (same as Qwen3.6). the -rea off server flag is mandatory. once disabled, the model is fast (42 tok/s) and produces clean direct output.

deep-dive benchmark: gpt-oss-20b (8 additional prompts)

after the initial comparison, we ran 8 more prompts through gpt-oss-20b to stress-test reliability across different coding skills.

result: 8/8 pass. combined with the 2 original tasks (portscout + logpulse), gpt-oss-20b completed 10/10 agentic coding tasks on 8GB VRAM.

consistent patterns:

• —self-correction works: 7 total self-fixes across 8 tasks. model finds its own bugs and fixes them.
• —context efficiency: 6-48% usage per task, no task exhausted the 32K window.
• —no hallucinated APIs: passed the stdlib-only trap cleanly.
• —edit tool weakness: exact string matching fails repeatedly, model falls back to full file rewrites.
• —directory scan waste: runs ls -R from home, dumping 11K+ lines into context (2 of 8 prompts).
• —rumination: debates obvious decisions, wasting tokens without causing failures.

verdict

gpt-oss-20b + Pi is the first viable local coding agent on 8GB VRAM. it completed 10/10 agentic coding tasks: the original easy + hard comparison tasks, plus 8 deep-dive prompts covering bug fixes, feature additions, TDD, data pipelines, hallucination traps, error recovery, multi-module projects, and bash scripting. the other configs all failed: 9B too small for tool calls, 35B Qwen3.6 too slow for agent loops.

key recipe:

• —model: gpt-oss-20b Q4KM (11 GB, 1.8 GB VRAM with ncmoe=30)
• —framework: Pi coding agent (simple read/write/edit/bash tools)
• —server: llama-server with --jinja, ncmoe=30, -c 32768, -n 8192
• —hardware: any 8GB GPU + 16GB+ system RAM

schema

GBNF structured CoT experiment (2026-05-21)

finding #3 above says "thinking mode is unusable for agentic work." can we fix that? the structured CoT approach (credit: andthattoo/structured-cot) uses a 4-rule GBNF grammar to force the model into a strict think-then-code pattern:

root  ::= think code
think ::= "<think>\n" "GOAL: " line "APPROACH: " line "EDGE: " line "</think>\n\n"
line  ::= [^\n]+ "\n"
code  ::= [\x09\x0A\x0D\x20-\x7E]+

the grammar constrains the full output stream (including reasoning_content). results on Qwen3.6-35B-A3B:

easy tasks: grammar vs free-form

free-form Qwen3.6 hit the token limit on all 3 tasks. it burned the entire 2048-token budget on reasoning and produced zero usable code on 2 of 3 tasks. the grammar compressed output 6.2x overall and every task produced correct, valid Python.

hard tasks: 3-field vs 5-field grammar

we also tested a 5-field variant (GOAL/STATE/ALGO/EDGE/VERIFY) on harder problems:

the 5-field grammar adds marginal value on complex tasks (caught a node-collection edge case on topo sort, used __slots__ on LRU cache) but the improvement doesn't justify the extra token cost for general use.

verdict

the 3-field grammar (GOAL/APPROACH/EDGE) is the sweet spot. it eliminates rumination, compresses tokens 4.6-11.4x, speeds up generation 5x, and produces valid code on 6/6 tasks. the model already knows the answers, it just needs to stop overthinking.

key caveat: this was tested on direct code generation, not agentic tool-call generation. applying grammar to an agent loop would require a grammar that handles both code and JSON tool calls.

related

• —inference speed benchmarks (same hardware): witcheer/windows-rtx-4060ti-8gb-moe-offload-bench-2026-05
• —model collection: 8GB VRAM local LLMs, practitioner-tested