Modelpublic

shreecloud/llama3.1-8b-lora-kaggle

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face

0likes8downloads

Model Card

🦙 Llama-3.1-8B LoRA Fine-tuned (Kaggle)

Fine-tuned Llama-3.1-8B using LoRA — trained on Kaggle's free T4 GPUs.

Model Details

—Base Model: NousResearch/Meta-Llama-3.1-8B
—Method: LoRA (PEFT)
—Rank (`r`): 8–16
—Trainable Parameters: ~0.04–0.12% of total
—Hardware: Kaggle Free Tier (Tesla T4)

Quick Inference

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

model_id = "shreecloud/llama3.1-8b-lora-kaggle"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype=torch.float16,
    load_in_4bit=True,
)

prompt = "### Instruction:\nWrite a professional email about...\n\n### Response:\n"

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))