astom-M/matsuo-llm-advanced-dbbench-v1

DBBench SQL Agent - Phase 1 (Multi-Turn Fine-Tuned)

Model Description

This is a fine-tuned Qwen2.5-7B-Instruct model optimized for DBBench SQL agent tasks in the Matsuo Lab LLM Course 2025 Advanced Competition.

Version: Phase 1 (DBBench specialized) Key Innovation: Multi-turn trajectory training with 3 patterns (direct, exploration, self-correction)

Training Details

• —Base Model: Qwen/Qwen2.5-7B-Instruct
• —Training Method: QLoRA (4-bit quantization + LoRA, merged)
• —LoRA Config: r=32, alpha=32, dropout=0, targetmodules=[qproj, kproj, vproj, oproj, gateproj, upproj, downproj]
• —Dataset: 7,490 multi-turn SQL trajectories (Spider train + BIRD mini_dev)
• —Training: 0.5 epoch, lr=5e-5, effective batch size=16
• —Final Train Loss: 0.22
• —Eval Loss: 0.235
• —Training Time: ~10 min (RTX 5090 + Unsloth)

Data Generation

Training data was generated using rule-based methods only (no LLM involvement):

• —Source: Spider train split (6,990 samples) + BIRD mini_dev (500 samples)
• —Patterns: Direct answer (40%), Exploration (30%), Self-correction (30%)
• —Schema: Auto-generated CREATE TABLE from SQL parsing (sqlparse)
• —Results: Template-based fake MySQL CLI output
• —Reasoning: Template-based English text (random.choice from predefined strings)
• —No LLM used for data generation, filtering, or quality assessment

Response Format

The model follows the DBBench protocol:

For SQL queries:

[Reasoning about the question]

Action: Operation

SELECT column FROM table WHERE condition;

For final answers:

[Reasoning about the result]

Action: Answer
Final Answer: ["ANSWER1", "ANSWER2", ...]

Local Test Results (5/5 Passed)

All 5 test queries produced valid format with:

• —Correct Action: Operation + SQL block structure
• —Single-line SQL statements
• —MySQL-compatible syntax with backtick quoting
• —Natural reasoning text before each action

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "astom-M/matsuo-llm-advanced-dbbench-v1"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    device_map="auto",
    torch_dtype="bfloat16"
)

Competition

Event: Matsuo Lab LLM Course 2025 Advanced Competition Target: DBBench task improvement (baseline 51.3% → target 60-70%) Training Date: February 2026

License

Same as base model (Qwen2.5-7B-Instruct) - Apache 2.0