Modelpublic

arif-butt/tinyllama-trl-lora-adapter

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face

0likes10downloads

Model Card

🦙 TinyLlama TRL LoRA Adapter - Parameter-Efficient Fine-Tuned Model

📋 Model Overview

This is a LoRA (Low-Rank Adaptation) adapter for TinyLlama (1.1B parameters) fine-tuned using TRL (Transformer Reinforcement Learning) framework. These are adapter weights only (~48 MB) that must be loaded on top of the base model. Perfect for sharing fine-tuned models without distributing the entire 2.2GB base model.

Key Features

Feature	Description
Parameter-Efficient	Only 48 MB vs 2.2 GB full model
Fast Download	Quick to download and upload
Easy Sharing	Share adapters without base model
Compatible	Works with any TinyLlama base model
Reusable	Multiple adapters can be swapped

Adapter Architecture

Component	Specification
Method	LoRA (Low-Rank Adaptation)
Rank (r)	16
Alpha	32
Scaling Factor	alpha/r = 2.0
Dropout	0.05
Trainable Parameters	3,670,016 (~0.19% of base)
Adapter Size	48 MB

🚀 Usage Guide

Installation

bash

pip install transformers peft accelerate torch

Method 1: Load with PEFT (Recommended)

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Load base model
BASE_MODEL = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
ADAPTER = "arif-butt/tinyllama-trl-lora-adapter"

print("Loading base model...")
base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    torch_dtype=torch.float16,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)

# Attach adapter
print("Attaching LoRA adapter...")
model = PeftModel.from_pretrained(base_model, ADAPTER)
model.eval()

# Test prompt
prompt = "Q: Name all the courses Arif butt teach?\nA:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=100,
        temperature=0.2,
        do_sample=True,
        pad_token_id=tokenizer.eos_token_id,
    )

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(f"Response: {response[len(prompt):].strip()}")



Method 2: Using Pipeline
from transformers import pipeline
from peft import PeftModel
import torch

base_model = AutoModelForCausalLM.from_pretrained(
    "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
    torch_dtype=torch.float16,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
model = PeftModel.from_pretrained(base_model, "arif-butt/tinyllama-trl-lora-adapter")

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
result = pipe("Q: What is machine learning?\nA:", max_new_tokens=100)
print(result[0]["generated_text"])


Load with 4-bit Quantization (QLoRA)
from transformers import BitsAndBytesConfig

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
)

base = AutoModelForCausalLM.from_pretrained(
    "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
    quantization_config=bnb_config,
    device_map="auto",
)
model = PeftModel.from_pretrained(base, "arif-butt/tinyllama-trl-lora-adapter")