ZennyKenny/GRPO_LoRA_Qwen_3B

Qwen2.5-3B-Instruct with LoRA Adapter

This model is a fine-tuned version of the unsloth/qwen2.5-3b-instruct-unsloth-bnb-4bit using Parameter-Efficient Fine-Tuning (PEFT) with the LoRA method.

Model Details

Model Description

This model applies LoRA (Low-Rank Adaptation) to the Qwen2.5-3B-Instruct base model, targeting key projection modules for efficient fine-tuning. It is quantized to 4-bit precision using the Unsloth library to optimize for inference performance on lower-resource hardware.

• —Developed by: montebello.ai
• —Funded by: Bootstrapped 4 Life
• —Model type: Causal Language Model with LoRA Adapter
• —Language(s) (NLP): English (primary), multilingual support for some other languages (research ongoing).
• —License: MIT
• —Finetuned from model: unsloth/qwen2.5-3b-instruct-unsloth-bnb-4bit

Model Sources

• —Demo [optional]: [COMING SOON]

Uses

Direct Use

This model is designed for natural language understanding and generation tasks, including:

• —Conversational AI
• —Summarization
• —Text completion
• —Question answering

Downstream Use

Fine-tuning the model for specific NLP tasks, such as custom domain conversational agents.

Out-of-Scope Use

• —Real-time critical systems requiring guaranteed safety and accuracy.
• —Generating content for sensitive domains without human oversight.

Bias, Risks, and Limitations

• —The base model may contain biases present in the original training data.
• —The model is not fine-tuned for safety-critical applications.
• —Limitations include possible hallucinations in generative outputs and language biases.

Recommendations

• —Perform task-specific evaluations before deploying the model.
• —Include human-in-the-loop for critical applications.

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

tokenizer = AutoTokenizer.from_pretrained("unsloth/qwen2.5-3b-instruct-unsloth-bnb-4bit")
base_model = AutoModelForCausalLM.from_pretrained("unsloth/qwen2.5-3b-instruct-unsloth-bnb-4bit")

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "path_to_your_adapter")

# Example inference
input_text = "Your input text here."
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0]))

Training Details

Training Procedure

• —Preprocessing: Tokenization with AutoTokenizer.
• —Training regime: bf16 mixed precision with LoRA fine-tuning.
• —LoRA Configuration:
• —lora_alpha: 64
• —r: 64
• —target_modules: ['vproj', 'oproj', 'upproj', 'gateproj', 'downproj', 'qproj', 'k_proj']
• —bias: none

Training Data

This model was trained on the GSM8K dataset from OpenAI.

Environmental Impact

• —Hardware Type: T4 TPU
• —Hours used: 2.5 hours

Technical Specifications

Model Architecture and Objective

• —Transformer-based causal language model using LoRA for efficient adaptation.

Compute Infrastructure

• —Hardware: [Specify GPU/CPU setup]
• —Software:
• —Python 3.x
• —Transformers library
• —PEFT 0.14.0

Citation [optional]

BibTeX:

@misc{your2025model,
  title={Qwen2.5-3B-Instruct with LoRA Adapter},
  author={Kenneth Hamilton},
  year={2025}
}

APA:

Your Name. (2025). Qwen2.5-3B-Instruct with LoRA Adapter. Retrieved from Your repository link.

Model Card Contact

• —Contact: [Your contact information]

Framework versions

• —PEFT 0.14.0