Modelpublic

SaniaKhalid/tinyllama-unsloth-merged

sourceHugging Faceapache-2.0updated 3d agoView on Hugging Face

0likes173downloads

Model Card

🦙 TinyLlama Unsloth Merged - Full Fine-tuned Model

Model Description

This is a fully merged model of TinyLlama (1.1B parameters) fine-tuned using Unsloth optimizations with LoRA adapters, then merged into a single complete model. Unlike adapter-only versions, this model is standalone and can be loaded without PEFT library.

Key Features:

—Fully Merged: No separate adapter files needed
—Unsloth Optimized: 2-3x faster inference with Unsloth kernels
—Memory Efficient: 30-50% less memory usage than standard models
—Standalone: Load directly with transformers or Unsloth
—Ready to Use: Single folder with model + tokenizer

Model Details:

—Base Model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
—Fine-tuning Method: LoRA with Unsloth optimizations (merged)
—Format: PyTorch safetensors
—Parameters: 1.1 Billion
—Context Length: 2048 tokens
—Precision: FP16 (float16)

🚀 Usage

Option 1: Using Transformers (Recommended)

python

# ── Load Merged Model with Transformers ───────────────────────────────────

from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
import torch

# Model identifier
MODEL_ID = "arif-butt/tinyllama-unsloth-merged"

print("Loading model and tokenizer...")
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True,
)
model.eval()
print("✅ Model loaded successfully!")

# Test prompt
prompt = "Q: Name all the courses Arif butt teach?\nA:"

# Tokenize
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

# Generate
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=100,
        temperature=0.2,
        do_sample=True,
        pad_token_id=tokenizer.eos_token_id,
    )

# Decode
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(f"Prompt: {prompt}")
print(f"Response: {response}")
Option 2: Using Pipeline
# ── Text Generation Pipeline ─────────────────────────────────────────────

from transformers import pipeline
import torch

MODEL_ID = "arif-butt/tinyllama-unsloth-merged"

pipe = pipeline(
    "text-generation",
    model=MODEL_ID,
    torch_dtype=torch.float16,
    device_map="auto",
)

prompt = "Q: What is machine learning?\nA:"
output = pipe(prompt, max_new_tokens=100, temperature=0.2)
print(output[0]["generated_text"])
Option 3: Using Unsloth (Faster Inference)
# ── Load with Unsloth for Maximum Performance ────────────────────────────

from unsloth import FastLanguageModel
import torch

MODEL_ID = "arif-butt/tinyllama-unsloth-merged"

print("Loading model with Unsloth...")
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name=MODEL_ID,
    max_seq_length=2048,
    dtype=torch.float16,
    device_map="auto",
)
print("✅ Model loaded with Unsloth optimizations!")

# Test prompt
prompt = "Q: Explain neural networks\nA:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=150,
    temperature=0.3,
    do_sample=True,
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(f"Response: {response}")

Fine-tuning Configuration
LORA_R       = 16        # Rank of LoRA matrices
LORA_ALPHA   = 32        # Scaling factor (alpha/r = 2.0)
LORA_DROPOUT = 0.05      # Dropout for regularization
TARGET_MODULES = [       # Layers where LoRA is applied
    "q_proj",            # Query projection
    "k_proj",            # Key projection  
    "v_proj",            # Value projection
    "o_proj",            # Output projection
    "gate_proj",         # Gate projection (MLP)
    "up_proj",           # Up projection (MLP)
    "down_proj"          # Down projection (MLP)
]