Avinaash/chartsense-sft-llama4-scout-lora

chartsense-sft-llama4-scout-lora

LoRA adapter produced by Adaption AutoScientist from the chartsense_sft dataset.

Provenance

The original base_model_name_or_path pointed at togethercomputer/Llama-4-Scout-17B-16E-Instruct_bnb_4bit, a Together internal repo that is not publicly readable. It has been repointed to meta-llama/Llama-4-Scout-17B-16E-Instruct, which is gated and needs licence approval on your account.

Trained as QLoRA against a bitsandbytes 4-bit base. Load the base in 4-bit (bnb nf4) to match training conditions.

Training curve

Final eval loss 0.8642578125, from trainer_state.json (included). The curve was still descending at the last step, so this is not trained to convergence.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-4-Scout-17B-16E-Instruct", device_map="auto", load_in_4bit=True,
)
model = PeftModel.from_pretrained(base, "Avinaash/chartsense-sft-llama4-scout-lora")
tok = AutoTokenizer.from_pretrained("Avinaash/chartsense-sft-llama4-scout-lora")