sirunchained/text-to-sql-model-v2

Text-to-SQL Model v2

🚀 Model Description

This is Version 2 of the sirunchained/text-to-sql-model, fine-tuned from google/gemma-3-270m-it for Text-to-SQL generation. In this version, the model is merged with the LoRA adapter – you can load it directly with pipeline() (no PEFT required).

Key improvements in v2:

🧠 Task

Text-to-SQL Generation Converts natural language questions into SQL queries. Supports:

• —✅ SELECT queries (with JOINs, aggregations, subqueries)
• —✅ INSERT operations
• —✅ UPDATE operations
• —✅ DELETE operations (currently weak at this)

📊 Training Details

📈 Training Performance

Best validation loss was achieved at epoch 4 & 5 (0.640). Highest mean token accuracy on validation was at epoch 4 (85.1%).

💻 Quick Start

Using Pipeline (Recommended)

from transformers import pipeline

generator = pipeline(
    "text-generation",
    model="sirunchained/text-to-sql-model-v2",
    device=0  # or "cuda"
)

# Example with schema
prompt = """<start_of_turn>user
# Schema
customers(id, name, email, country)
# Text
Find customers from USA.<end_of_turn>
<start_of_turn>model
"""
result = generator(prompt, max_new_tokens=128)
print(result[0]["generated_text"])

With Chat Template

from transformers import pipeline

pipe = pipeline("text-generation", model="sirunchained/text-to-sql-model-v2")

messages = [
    {"role": "user", "content": "# Schema\ncustomers(id, name, email)\n\n# Text\nFind customers with gmail emails."}
]

outputs = pipe(
    pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True),
    max_new_tokens=128
)
print(outputs[0]["generated_text"])

🎯 Dataset

The model was trained on sirunchained/text-to-sql-dataset:

Dataset format:

• —text: Natural language question
• —schema: Optional database schema
• —query: Target SQL query

🧪 Evaluation Results

Test Set Performance (Epoch 5 model):

📝 Version History

🛠️ Training Configuration

# LoRA Configuration
LoraConfig(
    r=8,
    lora_alpha=16,
    lora_dropout=0.05,
    bias="none",
    task_type=TaskType.CAUSAL_LM,
)

# Training Configuration
SFTConfig(
    num_train_epochs=5,
    per_device_train_batch_size=32,
    learning_rate=5e-5,
    lr_scheduler_type="constant",
    weight_decay=0.0,
    load_best_model_at_end=True,
    metric_for_best_model="mean_token_accuracy",
    greater_is_better=True,
)

⚠️ Important Notes

• —This is a small language model (270M parameters) – works on T4 GPUs
• —Provide schema only when needed – works with or without it
• —For non-SQL requests, the model outputs INVALID_QUERY (trained with negative samples)
• —The model handles INSERT, UPDATE, and DELETE queries correctly

🔗 Links

• —Base Model: google/gemma-3-270m-it
• —Dataset: sirunchained/text-to-sql-dataset
• —v1 (Adapter): sirunchained/text-to-sql-model

🙏 Acknowledgments

Built with:

• —Hugging Face Transformers
• —TRL (Transformer Reinforcement Learning)
• —PEFT (Parameter-Efficient Fine-Tuning)
• —bitsandbytes
• —Gradio for the demo interface which you can use here

📄 License

This model is released under the same license as Google's Gemma model. See the Gemma model card for details.