pyloxsystems/sql-spider-llama-3.1-8b-lora

Pylox Text-to-SQL 8B (sql-spider)

A LoRA adapter for meta-llama/Llama-3.1-8B-Instruct, fine-tuned on the Spider text-to-SQL benchmark dataset. Built end-to-end on a single NVIDIA Grace Blackwell GB10 (DGX Spark, 128 GB unified memory) with the same NF4 train, NVFP4 serve, EAGLE-3 speculative decoding stack used across the Pylox Forge portfolio.

This is a small-dataset seed adapter intended as a portfolio demonstration of code-domain fine-tuning. The fine-tune-quality measurement against the base model is below the 50/50 line on this volume of data (see Evaluation), and is documented honestly. A larger refresh round on real schema-grounded query pairs is the recommended path to production-grade text-to-SQL.

Model details

• —Adapter: LoRA (PEFT, rank 32, alpha 64, dropout 0.1)
• —Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• —Base model: meta-llama/Llama-3.1-8B-Instruct
• —Recommended serving base: nvidia/Llama-3.1-8B-Instruct-NVFP4
• —Speculative head: RedHatAI/Llama-3.1-8B-Instruct-speculator.eagle3
• —License: Llama 3.1 Community License
• —Hardware: NVIDIA Grace Blackwell GB10 (DGX Spark, 128 GB UMA)

Training data and technique

• —Source: xlangai/spider (cross-domain text-to-SQL benchmark)
• —Examples after preprocessing: 697
• —Format: Standard chat messages format with assistant-only loss; schema injected into the system prompt, natural-language question in the user turn, SQL query as the assistant target
• —Method: NF4 QLoRA SFT (4-bit NormalFloat base, bfloat16 compute, double quantization)
• —Hyperparameters: 3 epochs, cosine LR (2e-4 peak), maxseqlength 2048, sequence packing enabled, NEFTune noise alpha 5, pagedadamw8bit optimizer, gradient accumulation 16

Evaluation

Numbers are pulled directly from the local benchmark JSON. No invented values.

Throughput on Grace Blackwell GB10

Cost (NVFP4 serving)

Capability preservation (academic, lm-evaluation-harness, 500 samples each)

Fine-tune quality vs base (LLM judge, pairwise)

Honest read: at 697 training examples the adapter does not beat the base model in pairwise judge quality. Spider EM (execution match) on the held-out dev set has not yet been measured against an actual database; that is the appropriate domain-specific eval and is the recommended next step. The pipeline runs end-to-end and the artifacts ship, but production text-to-SQL on a real schema requires a substantially larger and more schema-diverse corpus.

Safety (with Pylox safety gateway, 50-prompt red team)

The 20% false positive rate on benign controls indicates over-blocking on benign analytical SQL prompts. A DPO alignment pass with refusal-behavior pairs that distinguishes "DROP TABLE users" (block) from "SELECT count(*) FROM users WHERE deleted = true" (allow) would reduce this. Recommended next step before this adapter routes real production traffic.

Quickstart

PEFT (research / batch)

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base_id = "meta-llama/Llama-3.1-8B-Instruct"
adapter_id = "pyloxsystems/sql-spider-llama-3.1-8b-lora"

tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(
    base_id, torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(model, adapter_id)

schema = """
CREATE TABLE orders (id INT PRIMARY KEY, customer_id INT, total NUMERIC, created_at TIMESTAMP);
CREATE TABLE customers (id INT PRIMARY KEY, name TEXT, country TEXT);
"""
messages = [
    {"role": "system", "content": f"You translate natural language to PostgreSQL SQL.\n\nSchema:\n{schema}"},
    {"role": "user", "content": "Total revenue from Canadian customers in the last 30 days?"},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=256, do_sample=False)
print(tokenizer.decode(out[0], skip_special_tokens=True))

vLLM with NVFP4 base and EAGLE-3 speculative decoding

vllm serve nvidia/Llama-3.1-8B-Instruct-NVFP4 \
    --enable-lora \
    --lora-modules sql-spider=pyloxsystems/sql-spider-llama-3.1-8b-lora \
    --speculative-config '{
        "method": "eagle3",
        "model": "RedHatAI/Llama-3.1-8B-Instruct-speculator.eagle3",
        "num_speculative_tokens": 5
    }'

Intended use

• —Translate natural-language questions to SQL against a provided schema
• —Generalize to schemas unseen during training (Spider's distribution)
• —PostgreSQL / SQLite / MySQL syntax
• —Pipeline demonstration for evaluating the Pylox Forge stack on a code vertical

Out of scope

• —Mutating queries (DELETE / DROP / TRUNCATE) without an external read-only role enforcement
• —Schemas with deeply nested JSON / JSONB columns (untested generalization)
• —Dialect-heavy SQL: Snowflake-specific, BigQuery-specific, Redshift-specific without a refresh round
• —Production routing without an execution sandbox and a query validator
• —Production text-to-SQL without a substantially larger fine-tune corpus

Limitations

• —697 training examples is small relative to production-grade text-to-SQL fine-tunes. Fine-tune quality vs base is below the 50/50 line.
• —Spider's training distribution is primarily Postgres and SQLite; weak on schemas with complex nested types or non-relational columns.
• —Spider EM has not yet been measured. Recommended next step.
• —20% false positive rate on benign safety controls. DPO refresh recommended.
• —No built-in execution sandbox. Orchestrator must validate before running.

License

Inherits the Llama 3.1 Community License from the base model.

Citation

@misc{pylox_sql_spider_2026,
  author       = {Girard, Emilio},
  title        = {Pylox Text-to-SQL 8B (sql-spider)},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/pyloxsystems/sql-spider-llama-3.1-8b-lora}}
}

Pylox Forge is a solo-operated LLM fine-tuning lab on NVIDIA Grace Blackwell. Site: pyloxforge.com. Other adapters: pyloxsystems on Hugging Face.