mehmet1899/llama32-3b-instruct-nl2sql-lora

Llama 3.2 3B Instruct NL2SQL LoRA

This repository contains the final LoRA adapter for the Llama 3.2 3B Instruct model line used in a master's thesis project on local large language models for NL2SQL generation.

The adapter generates SQLite queries from natural-language questions and a relational database schema. It must be loaded together with the corresponding base model.

Base model

• —Model: meta-llama/Llama-3.2-3B-Instruct
• —Revision: 0cb88a4f764b7a12671c53f0838cd831a0843b95
• —Access to the base model may require acceptance of Meta's license terms on Hugging Face.

Adapter

• —Method: LoRA supervised fine-tuning
• —LoRA rank: 8
• —LoRA alpha: 16
• —LoRA dropout: 0.05
• —Target modules: all suitable linear modules
• —Quantization during training: none
• —Maximum training sequence length: 2,048 tokens
• —Best checkpoint: checkpoint-509
• —The published root adapter corresponds to the selected best checkpoint.

SHA-256 of adapter_model.safetensors:

fcd4241f7a2e8e0388f13f0dd9517486cbee43fc3169c983a54e7b716c0e502d

Training configuration

• —Training examples: 25,000
• —Spider Train examples: 6,960
• —SQL Create Context examples: 18,040
• —Validation set: MixedVal2500-v2
• —Validation examples: 2,500
• —Learning rate: 1e-4
• —Scheduler: constant
• —Train batch size: 2
• —Gradient accumulation steps: 4
• —Effective batch size: 8
• —Seed: 42
• —Maximum epochs: 5
• —Early stopping patience: 2
• —Early stopping threshold: 0.001
• —Precision: FP16
• —Gradient checkpointing: enabled
• —Attention implementation: FlashAttention 2

Spider Dev was not used for training, validation, early stopping, or checkpoint selection.

Evaluation

The final adapter was evaluated on all 1,032 Spider Dev cases.

Zero-shot evaluation:

• —Execution Match Accuracy: 61.05% (630/1,032)
• —Execution Success Rate: 86.82% (896/1,032)
• —Maximum input length: 2,048 tokens
• —Maximum generated tokens: 256

The corresponding starting model achieved an Execution Match Accuracy of 55.04% (568/1,032) under the same zero-shot evaluation condition.

Intended prompt behavior

The model is instructed to return only a valid SQLite query:

• —no explanation
• —no Markdown
• —no comments
• —no unnecessary tables or columns
• —only SELECT or WITH queries
• —output terminated with a semicolon

The native Llama chat template and the exact project-specific prompt construction are documented in the accompanying GitHub repository.

Loading

~~~python import torch from peft import PeftModel from transformers import AutoModelForCausalLM, AutoTokenizer

basemodelid = "meta-llama/Llama-3.2-3B-Instruct" adapterid = "mehmet1899/llama32-3b-instruct-nl2sql-lora" adapterrevision = "87afdd0c565da4570ebd129a4098f50719e0f76e"

tokenizer = AutoTokenizer.frompretrained( basemodel_id, revision="0cb88a4f764b7a12671c53f0838cd831a0843b95", )

model = AutoModelForCausalLM.frompretrained( basemodelid, revision="0cb88a4f764b7a12671c53f0838cd831a0843b95", torchdtype=torch.float16, device_map="auto", )

model = PeftModel.frompretrained( model, adapterid, revision=adapter_revision, ) model.eval() ~~~

Reproducibility

Code, training and evaluation configurations, environment information, run manifests, and result summaries are available at:

https://github.com/md181099/nl2sql-masterthesis

The files training_metadata.json, training_history.csv, and training_history.jsonl provide additional training provenance.

Limitations

• —The adapter was evaluated primarily on the Spider benchmark and SQLite databases.
• —Performance on other database systems or unseen schema conventions is not guaranteed.
• —Access to the base model is governed by Meta's model license.
• —Execution Match depends on the database contents and the execution-based evaluation procedure.
• —The model may still generate invalid, incomplete, or semantically incorrect SQL.
• —The adapter should not be used to execute unrestricted queries against production databases without validation and access controls.