Modelpublic

Legeng/Llama-3.1-8B-PubMedQA-QLoRA

sourceHugging Facellama3.1updated 3mo agoView on Hugging Face

2likes8downloads

Model Card

Llama-3.1-8B-PubMedQA-QLoRA

QLoRA adapter for Llama-3.1-8B-Instruct, fine-tuned on PubMedQA for yes / no / maybe biomedical question answering (run5).

Built with Llama. Use of the base model is governed by the Llama 3.1 Community License.

Model details

—Base model: unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit
—Method: QLoRA (4-bit base), LoRA r=16, alpha=32, dropout=0.0
—Target modules: q,k,v,o,gate,up,down projections
—Task: PubMedQA classification (yes / no / maybe)
—Trained with: Unsloth + PEFT 0.18.1

Evaluation (run5)

model	acc	macro_f1	weighted_f1	maybe_recall	ece	yn_cross
Run 3 TEST calibrated (ref)	0.7740	0.6160	0.7620	0.1640	0.1090	-
run5 VAL raw	0.7000	0.5632	0.7278	0.2000	0.0541	2
run5 VAL calibrated	0.6400	0.5444	0.7056	0.2000	0.0541	0
run5 TEST raw	0.7220	0.6214	0.7449	0.3818	0.0650	27
run5 TEST calibrated	0.6480	0.5936	0.7018	0.5818	0.0650	15

Reliability diagrams, confusion matrices, and calibration sweeps are included in the metrics_run5_m150/ folder of this repo.

Usage

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit"
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
model = PeftModel.from_pretrained(model, "Legeng/Llama-3.1-8B-PubMedQA-QLoRA")
tok = AutoTokenizer.from_pretrained("Legeng/Llama-3.1-8B-PubMedQA-QLoRA")

Framework versions

—PEFT 0.18.1