badtheorylabs/BTL-4

BTL-4

A 35B agentic reasoning model from Bad Theory Labs, fine-tuned from Ornith-1.0-35B on an execution-gated reasoning corpus.

Built for tool use, software engineering and long-horizon agent work.

Benchmarks

BFCL and LiveCodeBench were run in-house with the official scorers, full splits, no subsetting. The BFCL number is a paired comparison: identical harness, identical decoding, only the weights differ.

LiveCodeBench by difficulty

The set is 45% hard problems, which is what pulls the aggregate down.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "badtheorylabs/BTL-4"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16",
                                             device_map="auto")

messages = [{"role": "user", "content": "Refactor this function to be pure."}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True,
                                 return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=2048)
print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))

Serving

vllm serve badtheorylabs/BTL-4 \
  --max-model-len 131072 \
  --enable-auto-tool-choice --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3 \
  --trust-remote-code

llama.cpp, using the GGUF build:

llama-server -m BTL-4-IQ2_XXS.gguf --port 8080 \
  --jinja \
  --reasoning-format deepseek \
  -c 32768 -fa on \
  --cache-type-k q8_0 --cache-type-v q8_0

Reasoning must be separated from content, on every stack. The chat template strips reasoning from older turns, but only when the harness puts it in reasoning_content. With vLLM that is --reasoning-parser qwen3; with llama.cpp it is --reasoning-format deepseek. Without it, reasoning accumulates into content each turn and the model repeats turns instead of terminating.

Generation settings

Ornith's published settings, used for every number above:

Give it room to think. LiveCodeBench improved 60.9% → 66.1% purely by raising the output budget from 16K to 32K. At 16K, 23.5% of problems were truncated mid-solution and scored zero. Hard problems reason longer; cutting them off costs real points.

What it is good at

• —Tool calling — 73.5% BFCL v4 AST, +4.3 points over base
• —Competitive programming — 99.1% easy / 86.7% medium on LiveCodeBench v6
• —Long context — 262K native, and it uses it

What it is not

• —Not a chat model. It reasons before answering and is verbose by default.
• —Reasoning accumulates across agent turns. The chat template strips prior reasoning from older turns, but this only works if your harness separates it into reasoning_content. With vLLM, that means --reasoning-parser qwen3. Without it, thinking lands in content, accumulates every turn, and long agent runs degrade.
• —Token-hungry on hard problems. Budget accordingly.

Training

Fine-tuned from Ornith-1.0-35B on an execution-gated reasoning corpus: candidate trajectories were kept only where the resulting code actually ran and passed its tests, so the reasoning that survived is reasoning that led somewhere.

Citation

@misc{btl4-2026,
  title  = {BTL-4: An Execution-Gated Agentic Reasoning Model},
  author = {Bad Theory Labs},
  year   = {2026},
  url    = {https://huggingface.co/badtheorylabs/BTL-4}
}