imuki04/Qwen2.5-0.5B-OpenBookQA-QLoRA-Finetuned

Qwen2.5-0.5B - OpenBookQA QLoRA Fine-Tuned

This model is a Parameter-Efficient Fine-Tuned (PEFT) adapter version of the Qwen/Qwen2.5-0.5B causal language model on the allenai/openbookqa dataset. It utilizes QLoRA (Quantized Low-Rank Adaptation) to achieve high performance while minimizing compute and VRAM requirements.

Model Description

• —Model Type: Causal Language Model (with LoRA adapters)
• —Language: English
• —Base Model: Qwen/Qwen2.5-0.5B
• —Training Paradigm: QLoRA (4-bit Normalized Float Base Model + fp16 LoRA Adapters)
• —Task: Multiple-Choice Question Answering (MCQA)

Dataset and Prompt Format

The model was fine-tuned on the elementary science multiple-choice dataset, OpenBookQA.

The prompt requires a specific layout to output the exact multiple-choice letter:

Prompt Template:

Question: {question_stem}
A. {choice_1}
B. {choice_2}
C. {choice_3}
D. {choice_4}
Answer: 

Example Input for Inference:

Question: The sun is responsible for
A. puppies learning new tricks
B. children growing up and getting old
C. flowers wilting in a vase
D. plants sprouting, blooming and wilting
Answer: 

Training Hyperparameters (QLoRA)

This model leveraged parameter-efficient fine-tuning, injecting rank decomposition matrices into the bfloat16 representation while keeping the core base model frozen in 4-bit nf4 format.

LoRA Configuration:

• —Rank (r): 16
• —Alpha: 32
• —Target Modules: All linear layers (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj)
• —Dropout: 0.05

SFT Configuration:

• —Optimizer: paged_adamw_32bit
• —Epochs: 2
• —Learning Rate: 2e-4
• —Scheduler: Cosine
• —Warmup Ratio: 0.1 (10% steps)
• —Batch Size: 16 (Per-Device: 4, Gradient Accumulation: 4)
• —Quantization: bitsandbytes (loadin4bit=True, double quant=True)

Limitations and Bias

Like the full fine-tuned sibling, this model specializes strictly in OpenBookQA templates. Because it is PEFT-based, it retains the fundamental biases and reasoning framework of the underlying pre-trained Qwen2.5-0.5B base while adapting closely to the format specified above.

Usage

To load this PEFT model properly, you need peft, transformers, and bitsandbytes installed.

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

base_model_id = "Qwen/Qwen2.5-0.5B"
adapter_id = "imuki04/Qwen2.5-0.5B-OpenBookQA-QLoRA-Finetuned" # Replace your repo name if needed

# 1. Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(adapter_id)

# 2. Config to load base model in 4-bit
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

# 3. Load base model + attach adapters
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    quantization_config=bnb_config,
    device_map="auto"
)
model = PeftModel.from_pretrained(base_model, adapter_id)

# 4. Generate
prompt = "Question: The sun is responsible for\nA. puppies learning new tricks\nB. children growing up and getting old\nC. flowers wilting in a vase\nD. plants sprouting, blooming and wilting\nAnswer:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=5, temperature=0.0)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))