delimitter/qwen2.5-0.8b-synoema-tools-v1

Qwen2.5-Coder-0.8B — Synoema Tools v1

LoRA adapter fine-tuned on Qwen2.5-Coder-0.8B-Instruct for agentic tool-use with the Synoema MCP server.

Score: 92.9% (26/28 tasks) on the Synoema MCP agentic evaluation benchmark.

What is Synoema?

Synoema is an LLM-native programming language and runtime:

• —BPE-aligned operators — every operator maps to exactly 1 cl100k_base token
• —GBNF grammar for constrained decoding (structural correctness guarantee)
• —Cranelift JIT + WebAssembly compilation targets
• —MCP server exposing file_write, file_read, sno_typecheck, sno_run, search_corpus tools
• —Contract annotations (requires/ensures) for formal verification

Model Details

Training Approach: Carousel Fine-tuning

This model was trained using a carousel strategy:

Base model → C1 (eval) → C2 (eval) → ... → C1 (best: 92.9%)
                                             ↑ always from best adapter

Each cycle:

1. 1.Merge corpus — base corpus + all targeted examples for failing tasks
2. 2.Train 3 epochs from the best previous adapter
3. 3.Eval on 28 agentic tasks (real tool calls, real typecheck)
4. 4.Analyze failures → generate targeted examples → add to corpus
5. 5.Repeat from best adapter

Evaluation: 28 Agentic Tasks

The model is evaluated on real multi-turn tool-use scenarios. Each task requires calling MCP tools correctly in sequence. The eval runs actual sno typecheck and sno run commands — no mock results.

Result: 26/28 tasks passed (92.9%)

Remaining failures:

• —TU4 — must write if x > y then x else y, see typecheck error, then fix to ? x > y -> x : y (2-write pattern)
• —TU13 — same pattern with classify n = if n > 0 then 1 else 0 → ? n > 0 -> 1 : 0

Both require a strict write→typecheck→rewrite→typecheck sequence with exactly 2 file_write calls.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-0.8B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-0.8B-Instruct")
model = PeftModel.from_pretrained(base, "Delimitter/qwen2.5-0.8b-synoema-tools-v1")

With unsloth (recommended for inference):

from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="Delimitter/qwen2.5-0.8b-synoema-tools-v1",
    max_seq_length=1024,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

System prompt format (ChatML):

<|im_start|>system
You are an AI coding assistant for the Synoema programming language...
<|im_end|>
<|im_start|>user
Write a quicksort in Synoema to src/qs.sno and run it.
<|im_end|>
<|im_start|>assistant

Corpus Composition

All examples validated with sno check + sno run before training.

Training History (Carousel)

C1 was selected as best before catastrophic forgetting set in at C9.

Synoema Language Quick Reference

-- Ternary (no if/else!)
max x y = ? x > y -> x : y

-- Pattern matching
fact 0 = 1
fact n = n * fact (n - 1)

-- List comprehension  
evens = [x | x <- [1..20], x % 2 == 0]

-- Space-separated lists (NOT commas)
main = qsort [3 1 4 1 5 9]

-- ADT
Shape = Circle Int | Rect Int Int
area (Circle r) = 3 * r * r

License

Apache 2.0 — same as Qwen2.5-Coder base model.

Synoema is © Andrey Bubnov. See synoema.tech.