Modelpublic

chauben/advisorai-llama2-7b-stevens

sourceHugging Facellama2updated 6mo agoView on Hugging Face

0likes22downloads

Model Card

AdvisorAI — LLaMA-2-7B Fine-Tuned on Stevens Q&A Data

A QLoRA fine-tuned version of LLaMA-2-7B adapted for academic advising at Stevens Institute of Technology, trained on 87,782 domain-specific Q&A pairs.

Training Details

Parameter	Value
Base Model	meta-llama/Llama-2-7b-hf
Method	QLoRA (4-bit NF4)
LoRA Rank / Alpha	16 / 32
Target Modules	qproj, kproj, vproj, oproj, gateproj, upproj, down_proj
Epochs	6
Learning Rate	2e-4
Effective Batch Size	32
Best Checkpoint	Step 7,500
Training Platform	Google Colab GPU

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig from peft import PeftModel import torch

bnbconfig = BitsAndBytesConfig( loadin4bit=True, bnb4bitquanttype="nf4", bnb4bitcompute_dtype=torch.float16 )

base = AutoModelForCausalLM.frompretrained( "meta-llama/Llama-2-7b-hf", quantizationconfig=bnbconfig, devicemap="auto" )

model = PeftModel.frompretrained(base, "nitinchaube/advisorai-llama2-7b-stevens") tokenizer = AutoTokenizer.frompretrained("nitinchaube/advisorai-llama2-7b-stevens")

prompt = """### Context: {paste stevens context here}

Question:

What are the concentrations in the Applied Mathematics program?

Answer:"""

inputs = tokenizer(prompt, returntensors="pt").to("cuda") output = model.generate(**inputs, maxnewtokens=256, temperature=0.7) print(tokenizer.decode(output[0], skipspecial_tokens=True))