Modelpublic

oliverkinch/qwen3.5-9b-text2sql-grpo

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face

0likes9downloads

Model Card

qwen3.5-9b-text2sql-grpo

An agentic text-to-SQL model: given a natural-language question and a SQLite database, it explores the schema with tools, runs candidate queries, observes real execution feedback, and submits a final SQL query.

Trained with agentic GRPO directly from vanilla `Qwen/Qwen3.5-9B` — no supervised finetuning stage, no teacher traces. 200 GRPO steps in an execution-grounded environment (verifiers / prime-rl): the reward executes the submitted query against the real database and compares result sets to gold (1.0 match / 0.1 runs-but-wrong / 0.0 error / −1.0 write attempt). Training prompts come from the BIRD train split, filtered to those the base policy solves 1–7 times out of 8 (prompts with no gradient signal are dropped).

Results (BIRD dev, 1,534 examples, execution accuracy)

setting	EX
agentic, greedy	63.4%
agentic, exec-consistency voting @ k=8 / k=32 (temp 1.0)	67.0% / 67.4%
single-turn (no tools), greedy	58.2%
base `Qwen/Qwen3.5-9B` zero-shot agentic, for reference	49.7%

Exec-consistency voting: sample k rollouts, group the submitted queries by the result set they return, exclude empty results, take the largest group (majority over result sets — never over SQL text, and without de-duplicating verbatim repeats).

Usage

The model calls tools in Qwen3.5's XML dialect — serve with vLLM using --enable-auto-tool-choice --tool-call-parser qwen3_xml (the hermes parser silently parses nothing).

It expects the agentic environment's tool set: inspect_schema, sample_rows, run_sql (read-only), submit. The training/eval harness, environment definition, and a ready-to-use CLI (scripts/ask.py, agentic and single-turn modes, works against any SQLite file) live in the training repository: https://github.com/oliverkinch/text2sql-env

It also works as a plain single-turn SQL generator (schema in the prompt, SQL out) at the lower accuracy listed above.

Limitations

—Tuned and measured on SQLite only.
—Rewarded only for read queries; it was penalized for writes, but do not point it at a database you cannot afford to mutate without a read-only connection (the training harness enforced one).
—BIRD dev was used for model selection across the project's history; treat dev numbers as development metrics, not test claims.