jk200201/qwen2.5-coder-7b-bird-cot-GGUF

Qwen2.5-Coder-7B BIRD CoT (Text-to-SQL) GGUF

GGUF quantizations of `jk200201/qwen2.5-coder-7b-bird-cot`, a 7B text-to-SQL model that reasons step by step over a database schema and then writes the SQL. Runs locally on CPU or a single consumer GPU with llama.cpp or Ollama.

On BIRD dev (realistic, messy schemas, the harder text-to-SQL benchmark) the full-precision model reaches 52.1% greedy result accuracy and 58.5% with self-consistency (Best-of-N, K=8). With self-consistency it matches DeepSeek V4-Pro (1.6T parameters) at roughly 0.4% of the size, running locally.

Files

Start with Q4_K_M. Move up to Q5KM or Q8_0 if you have the memory and want to close the small accuracy gap.

Quick start (Ollama)

# Download the Q4_K_M file, then from the same directory:
ollama create bird-cot -f Modelfile
ollama run bird-cot

Modelfile:

FROM ./qwen2.5-coder-7b-bird-cot-Q4_K_M.gguf

TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Response }}<|im_end|>
"""

SYSTEM """You are an expert SQLite query writer. Reason step by step about the schema and the question, then output the final query in a fenced sql code block."""

PARAMETER temperature 0
PARAMETER num_ctx 8192
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"

Then send a schema and a question:

Given the database schema and question, work out the correct SQLite query step by step.

Database Schema:
CREATE TABLE singer (Singer_ID INT, Name TEXT, Age INT);

Question: How many singers are older than 40?

Think step by step, then give the final answer in a fenced sql block.

The model returns its reasoning followed by a fenced SQL block. Take the last SQL block as the query.

Quick start (llama.cpp)

./llama-cli -m qwen2.5-coder-7b-bird-cot-Q4_K_M.gguf -no-cnv --temp 0 -n 512 \
  -p $'<|im_start|>system\nYou are an expert SQLite query writer. Reason step by step about the schema and the question, then output the final query in a fenced sql code block.<|im_end|>\n<|im_start|>user\nGiven the database schema and question, work out the correct SQLite query step by step.\n\nDatabase Schema:\nCREATE TABLE singer (Singer_ID INT, Name TEXT, Age INT);\n\nQuestion: How many singers are older than 40?\n\nThink step by step, then give the final answer in a fenced sql block.<|im_end|>\n<|im_start|>assistant\n'

Results: BIRD dev (execution result accuracy)

All models use a single chain-of-thought sample unless noted, so the comparison is like for like.

Result accuracy is the fraction of queries whose SQL executes to the same rows as the gold query (BIRD's official execution metric). The reported figures are for the full-precision model. Quantization retains most of that accuracy; Q4KM is the recommended balance.

Training

Reasoning distillation (CoT-SFT). A strong teacher (Qwen3-Coder-480B) generated step-by-step CoT solutions on BIRD train. Only execution-verified-correct chains were kept (5,593 examples), then supervised fine-tuned into the 7B. Distilling the teacher's reasoning generalized across BIRD's cross-domain dev databases better than distilling SQL answers directly.

Config: QLoRA (4-bit NF4, LoRA rank 32, alpha 64), 2 epochs, learning rate 2e-4 cosine, max sequence length 8192.

Model family

Limitations

• —Tuned for BIRD-style analytic SQL over realistic schemas. Unusual dialects or domains may need adaptation. It emits SQLite dialect.
• —Greedy (52.1%) is the deployable single-shot number. The 58.5% figure needs K=8 self-consistency (roughly 8x inference).
• —This is a 7B model. Review generated SQL before running it on production data.

Acknowledgements

Base model: Qwen2.5-Coder by Alibaba Qwen. Teacher: Qwen3-Coder-480B. Benchmark: BIRD (bird-bench.github.io).