CoolFace
Modelpublic

vishnuamarapu/Full-Fine-Tuning-Qwen-2.5-0.5B-instruct-LORA

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face
0likes
Model Card

license: apache-2.0 language:

  • —en pipelinetag: text-generation libraryname: transformers base_model: Qwen/Qwen2.5-0.5B-Instruct tags:
  • —qwen
  • —llm
  • —sft
  • —lora
  • —peft
  • —conversational
  • —transformers
  • —pytorch ---

LoRA Fine-Tuned Qwen2.5-0.5B-Instruct

This repository contains a LoRA (Low-Rank Adaptation) fine-tuned version of Qwen2.5-0.5B-Instruct. The repository stores only the LoRA adapter weights, making it lightweight and easy to share. To use this model, first load the base model and then apply the LoRA adapter.

Model Details

  • —Base Model: Qwen/Qwen2.5-0.5B-Instruct
  • —Fine-Tuning Method: LoRA (PEFT)
  • —Framework: Hugging Face Transformers + PEFT
  • —Task: Conversational Text Generation

Installation

bash
pip install transformers peft accelerate torch safetensors

Loading the Model

python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

BASE_MODEL = "Qwen/Qwen2.5-0.5B-Instruct"
LORA_MODEL = "vishnuamarapu/Full-Fine-Tuning-Qwen-2.5-0.5B-instruct-LORA"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)

base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
    device_map="auto"
)

model = PeftModel.from_pretrained(
    base_model,
    LORA_MODEL
)

model.eval()

Example Inference

python
messages = [
    {
        "role": "system",
        "content": (
            "You are Vishnu's personal AI assistant. "
            "Answer questions about Vishnu."
        )
    },
    {
        "role": "user",
        "content": "Tell me about Vishnu."
    }
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(
    text,
    return_tensors="pt"
).to(model.device)

with torch.no_grad():

    outputs = model.generate(
        **inputs,
        max_new_tokens=256,
        temperature=0.7,
        do_sample=True,
        top_p=0.9,
        repetition_penalty=1.1,
    )

response = tokenizer.decode(
    outputs[0][inputs.input_ids.shape[-1]:],
    skip_special_tokens=True,
)

print(response)

Gradio Demo

python
import torch
import gradio as gr

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

BASE_MODEL = "Qwen/Qwen2.5-0.5B-Instruct"
LORA_MODEL = "vishnuamarapu/Full-Fine-Tuning-Qwen-2.5-0.5B-instruct-LORA"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)

base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
    device_map="auto"
)

model = PeftModel.from_pretrained(
    base_model,
    LORA_MODEL
)

model.eval()

SYSTEM_PROMPT = (
                """ You are Vishnu's personal AI assistant.
Always answer using the information you have learned about Vishnu.
Do not invent facts.
If you do not know the answer, say you don't know.
Answer as Vishnu in first person."""
)

def generate(message, history):

    messages = [
        {
            "role": "system",
            "content": SYSTEM_PROMPT,
        }
    ]

    for user, assistant in history:
        messages.append({"role": "user", "content": user})
        messages.append({"role": "assistant", "content": assistant})

    messages.append({"role": "user", "content": message})

    text = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True,
    )

    inputs = tokenizer(
        text,
        return_tensors="pt",
    ).to(model.device)

    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=256,
            temperature=0.7,
            do_sample=True,
            top_p=0.9,
            repetition_penalty=1.1,
        )

    answer = tokenizer.decode(
        outputs[0][inputs.input_ids.shape[-1]:],
        skip_special_tokens=True,
    )

    return answer

gr.ChatInterface(
    fn=generate,
    title="Vishnu Personal AI",
    description="LoRA Fine-Tuned Qwen2.5-0.5B-Instruct",
).launch()

Generation Parameters

ParameterValue
maxnewtokens256
temperature0.7
top_p0.9
repetition_penalty1.1
device_mapauto
torch_dtypefloat16 (GPU) / float32 (CPU)

Repository Structure

adapter_config.json
adapter_model.safetensors
chat_template.jinja
tokenizer.json
tokenizer_config.json
README.md

Training Overview

This model was fine-tuned using LoRA (Low-Rank Adaptation) through the PEFT library. Only a small set of trainable adapter weights were updated while the base model remained frozen, resulting in a lightweight adapter that can be merged with the original base model for inference.

Citation

If you use this model in your work, please cite this repository.

bibtex
@misc{vishnu_qwen25_lora,
  author = {Vishnu Amarapu},
  title = {LoRA Fine-Tuned Qwen2.5-0.5B-Instruct},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/vishnuamarapu/Full-Fine-Tuning-Qwen-2.5-0.5B-instruct-LORA}}
}

Acknowledgements

  • —Alibaba Cloud Qwen Team for the base model.
  • —Hugging Face Transformers.
  • —PEFT (Parameter-Efficient Fine-Tuning).
  • —PyTorch.