poseidon1113/salesforce-codet5p-220m-sql-generator

language: en tags:

• —text-to-sql
• —codet5
• —fine-tuned
• —sql-generation datasets:
• —b-mc2/sql-create-context ---

CodeT5+ 220M — SQL Query Generator

Fine-tuned CodeT5+ 220M on the sql-create-context dataset to generate SQL queries from natural language questions and table schemas.

Training

• —Base model: Salesforce/codet5p-220m (220M parameters)
• —Dataset: b-mc2/sql-create-context (55k train / 23k val)
• —Method: Full fine-tuning with schema-aware prompting
• —Optimizer: AdamW, lr=3e-4, 5 epochs

Usage

from transformers import T5ForConditionalGeneration, AutoTokenizer
import torch

model = T5ForConditionalGeneration.from_pretrained("your-username/codet5p-sql-generator")
tokenizer = AutoTokenizer.from_pretrained("your-username/codet5p-sql-generator")

context = "CREATE TABLE employees (id INT, name VARCHAR, salary INT)"
question = "What is the average salary?"

src = f"Schema: {context}\nQuestion: {question}"
inputs = tokenizer(src, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=128, num_beams=4)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Improvements over GPT-2 baseline

• —Schema-aware prompting eliminates column/table hallucination
• —Encoder-decoder architecture suited for seq2seq SQL generation
• —10x more training examples (78k vs 7k)
• —max_length=512 prevents schema truncation