aravula7/qwen-sql-finetuning

1---2language: en3pipeline_tag: text-generation4library_name: transformers5tags:6- text-to-sql7- sql8- postgresql9- qwen2.510- qlora11- peft12- quantization13base_model: Qwen/Qwen2.5-3B-Instruct14license: mit15metrics:16- accuracy17---18 19# Qwen2.5-3B Text-to-SQL (PostgreSQL) — Fine-Tuned20 21## Overview22 23This repository contains a fine-tuned **Qwen/Qwen2.5-3B-Instruct** model specialized for **Text-to-SQL** generation in **PostgreSQL** for a realistic e-commerce + subscriptions analytics schema.24 25Artifacts are organized under a single Hub repo using subfolders:26 27* `fp16/` — merged FP16 model (recommended)28* `int8/` — quantized INT8 checkpoint (smaller footprint)29* `lora_adapter/` — LoRA adapter only (for further tuning / research)30 31## Intended use32 33**Use cases**34 35* Convert natural language questions into PostgreSQL queries.36* Analytical queries over common e-commerce tables (customers, orders, products, subscriptions) plus ML prediction tables (churn/forecast).37 38**Not for**39 40* Direct execution on sensitive or production databases without validation (schema checks, allow-lists, sandbox execution).41* Security-critical contexts (SQL injection prevention and access control must be handled outside the model).42 43## Training summary44 45| Item | Value |46| --- | --- |47| Base model | Qwen/Qwen2.5-3B-Instruct |48| Fine-tuning method | QLoRA (4-bit) |49| Optimizer | paged\_adamw\_8bit |50| Epochs | 4 |51| Training time | ~4 minutes (A100) |52| Trainable params | 29.9M (1.73% of 3B total) |53| Decoding | Greedy |54| Tracking | MLflow (DagsHub) |55 56## Evaluation summary (100 test examples)57 58Primary metric: **parseable PostgreSQL SQL** (validated with `sqlglot`).  59Secondary metric: **exact match** (strict string match vs. reference SQL).60 61| Model | Parseable SQL | Exact match | Mean latency (s) | P50 (s) | P95 (s) |62| --- | --- | --- | --- | --- | --- |63| **qwen\_finetuned\_fp16\_strict** | **1.00** | **0.15** | **0.433** | 0.427 | 0.736 |64| qwen\_finetuned\_int8\_strict | 0.99 | 0.20 | 2.152 | 2.541 | 3.610 |65| qwen\_baseline\_fp16 | 1.00 | 0.09 | 0.405 | 0.422 | 0.624 |66| qwen\_finetuned\_fp16 | 0.93 | 0.13 | 0.527 | 0.711 | 0.739 |67| qwen\_finetuned\_int8 | 0.93 | 0.13 | 2.672 | 3.454 | 3.623 |68| gpt-4o-mini | 1.00 | 0.04 | 1.616 | 1.551 | 2.820 |69| claude-3.5-haiku | 0.99 | 0.07 | 1.735 | 1.541 | 2.697 |70 71**Key Findings:**72 73* **Strict prompting is critical**: Adding "Return ONLY the PostgreSQL query. Do NOT include explanations, markdown, or commentary" improved parseable rate from 93% to 100%74* **Fine-tuning improves accuracy**: Exact match increased from 9% (baseline) to 15% (fine-tuned), a **67% improvement**75* **Quantization trade-offs**: INT8 maintains accuracy (20% exact match, best across all models) with 50% memory reduction but shows 5x latency increase76* **Competitive with APIs**: Fine-tuned model achieves **4x better exact match** than GPT-4o-mini while maintaining comparable speed77 78## Results Visualization79 80![Model Comparison](https://github.com/aravula7/qwen-sql-finetuning/raw/main/images/results_comparison.png)81 82*Parseable SQL rate and exact match accuracy comparison across all 7 models.*83 84## How to load85 86### Load the merged FP16 model (recommended)87 88```python89from transformers import AutoModelForCausalLM, AutoTokenizer90 91repo_id = "aravula7/qwen-sql-finetuning"92 93tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder="fp16")94model = AutoModelForCausalLM.from_pretrained(95    repo_id, 96    subfolder="fp16",97    torch_dtype=torch.float16,98    device_map="auto"99)100```101 102### Load the INT8 model103 104```python105from transformers import AutoModelForCausalLM, AutoTokenizer106 107repo_id = "aravula7/qwen-sql-finetuning"108 109tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder="int8")110model = AutoModelForCausalLM.from_pretrained(111    repo_id, 112    subfolder="int8",113    device_map="auto"114)115```116 117### Load base model + LoRA adapter118 119```python120from transformers import AutoModelForCausalLM, AutoTokenizer121from peft import PeftModel122import torch123 124base_id = "Qwen/Qwen2.5-3B-Instruct"125repo_id = "aravula7/qwen-sql-finetuning"126 127tokenizer = AutoTokenizer.from_pretrained(base_id)128base = AutoModelForCausalLM.from_pretrained(129    base_id,130    torch_dtype=torch.float16,131    device_map="auto"132)133 134model = PeftModel.from_pretrained(base, repo_id, subfolder="lora_adapter")135```136 137## Example inference138 139Below is a minimal example that encourages **SQL-only** output (critical for 100% parseability).140 141```python142import torch143from transformers import AutoModelForCausalLM, AutoTokenizer144 145repo_id = "aravula7/qwen-sql-finetuning"146tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder="fp16")147model = AutoModelForCausalLM.from_pretrained(148    repo_id, 149    subfolder="fp16",150    torch_dtype=torch.float16,151    device_map="auto"152)153 154system = "Return ONLY the PostgreSQL query. Do NOT include explanations, markdown, code fences, or commentary."155schema = "Table: customers (customer_id, email, state)\nTable: orders (order_id, customer_id, order_timestamp)"156request = "Show the number of orders per customer in 2025."157 158prompt = f"""{system}159 160Schema:161{schema}162 163Request:164{request}165"""166 167inputs = tokenizer(prompt, return_tensors="pt").to(model.device)168with torch.no_grad():169    out = model.generate(170        **inputs, 171        max_new_tokens=256, 172        do_sample=False,173        pad_token_id=tokenizer.eos_token_id174    )175 176sql = tokenizer.decode(out[0], skip_special_tokens=True)177# Extract SQL after prompt178sql = sql.split("Request:")[-1].strip()179print(sql)180```181 182## License183 184This project is licensed under the MIT License. The fine-tuned model is a derivative of Qwen2.5-3B-Instruct and inherits its license terms.185 186**Full documentation and code:** [GitHub Repository](https://github.com/aravula7/qwen-sql-finetuning)187 188## Reproducibility189 190Training and evaluation were tracked with MLflow on DagsHub. The GitHub repository contains:191 192* Complete Colab notebook with training and evaluation code193* Dataset (500 examples: 350 train, 50 val, 100 test)194* Visualization scripts for 3D performance analysis195* Production-ready inference code with error handling196 197**Links:**198* [GitHub Repository](https://github.com/aravula7/qwen-sql-finetuning)199* [MLflow Experiments](https://dagshub.com/aravula7/llm-finetuning)200* [Base Model](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct)201 202## Citation203 204```bibtex205@misc{qwen-sql-finetuning-2026,206  author = {Anirudh Reddy Ravula},207  title = {Qwen2.5-3B Text-to-SQL Fine-Tuning for PostgreSQL},208  year = {2026},209  publisher = {HuggingFace},210  howpublished = {\url{https://huggingface.co/aravula7/qwen-sql-finetuning}},211  note = {Fine-tuned with QLoRA for e-commerce SQL generation}212}213```