Modelpublic

VikramPal/mistral-7b-instruct-v0.3-lora

sourceHugging Faceapache-2.0updated 25d agoView on Hugging Face

0likes26downloads

Model Card

Mistral-7B-Instruct-v0.3 text-to-SQL LoRA

The rank-32 LoRA adapter that produced `VikramPal/mistral-7b-instruct-v0.3-bf16`, which is that adapter merged into the base model. Published separately so the fine-tune can be re-merged, inspected, or stacked onto a differently-quantized base without downloading 13.5 GiB.

This adapter is 335 611 085 B against the merge's 14 499 764 397 B, and it is the whole of the difference between them. Everything the fine-tune learned is here; nothing else was changed.

What it was trained on


Base	`mistralai/Mistral-7B-Instruct-v0.3`
Mixture	`gretelai/synthetic_text_to_sql` + `Salesforce/wikisql` + `b-mc2/sql-create-context`, 13 334 / 13 333 / 13 333 conversations
Kept	39 531 of 40 000; 469 dropped for exceeding 2048 tokens
Loss on	completion only (`mask_mode: template`), 1 426 125 supervised tokens of 15 858 075
Regime	LoRA r=32, alpha=64, dropout 0.05, on `q,k,v,o,gate,up,down`
Schedule	2 epochs, lr 1e-4, effective batch 32, 2472 steps
Final train loss	0.0540
Wall clock	3 h 49 m on one RTX PRO 6000 Blackwell

Decontaminated against the eval split before training: 4 gretel, 16 wikisql and 3342 create-context conversations removed for overlapping an evaluation problem. create-context is a training-only source -- it contributes to the adapter and is scored on nothing.

What it scores

Evaluated on 2454 held-out problems drawn equally from gretel, wikisql and spider, 2-shot, greedy, execution-free logic match:

78.16% overall (1918/2454), 0 unparseable, 0 truncated. By source:

source	accuracy	in the training mixture?
wikisql	93.89% (768/818)	yes
gretel	77.02% (630/818)	yes
spider	63.57% (520/818)	no

Spider is a third of the evaluation and none of the training mixture, so the 30-point gap between it and wikisql is what this adapter does not transfer. 1144 of the 1918 correct answers match the gold SQL as text; the other 774 are correct by execution equivalence.

Using it

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-Instruct-v0.3", dtype="bfloat16", device_map="auto"
)
model = PeftModel.from_pretrained(base, "VikramPal/mistral-7b-instruct-v0.3-lora")
tok = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.3")

Call model.merge_and_unload() to get the bf16 checkpoint published above, byte-for-byte modulo the merge's own float arithmetic.

The rest of the campaign

This adapter is one artifact of a quantization study. The others:

repo	what
`...-text2sql-bf16`	this adapter merged; the accuracy ceiling every quantized arm is measured against
`...-text2sql-DynQuant-4bit`	3.96 GB, 78.08%
`...-text2sql-DynQuant-3bit`	3.07 GB, 75.22%

The signals DynQuant allocates from -- per-module activation saliency and gradient plasticity -- were harvested during this LoRA run, by forward and backward hooks on 226 modules, and written alongside the adapter. That is the only reason the adapter and the quantization are the same campaign: the fine-tune is where the allocation's inputs come from.

—Code: <https://github.com/kambojvikram/dynquant>
—pip install dynquant

Limitations

Trained to emit a single SQL statement for a schema and a question, and nothing else. It is not a general assistant any more, and the 2-epoch schedule at loss 0.054 is well into the regime where it will answer off-task prompts in SQL. Outputs are not validated against a database and have not been checked for injection-safe parameterization -- do not execute them against anything you care about without review.