zaidrahim162/qwen2.5-3b-islamic-finance-qlora

Qwen2.5-3B Islamic Finance QLoRA

Model Description

This model is a QLoRA fine-tuned version of Qwen2.5-3B-Instruct specialized for answering questions related to Islamic Finance.

The model has been instruction-tuned on an Islamic finance dataset covering topics such as:

• —Murabaha
• —Musharakah
• —Mudarabah
• —Ijarah
• —Salam
• —Istisna
• —Sukuk
• —Takaful
• —Riba
• —Zakat
• —Shariah compliance
• —Islamic banking principles

The objective is to provide helpful, educational, and context-aware responses about Islamic finance.

Model Details

Developed by: Zaid Rahim

Base Model: Qwen/Qwen2.5-3B-Instruct

Fine-tuning Method:

• —QLoRA
• —PEFT
• —bitsandbytes 4-bit quantization

Framework

• —Hugging Face Transformers
• —PEFT
• —TRL

Language

• —English

Intended Uses

This model is intended for

• —Educational applications
• —Islamic finance assistants
• —Research
• —Retrieval-Augmented Generation (RAG)
• —Question answering

Limitations

This model:

• —is not a replacement for qualified Islamic scholars.
• —may generate incorrect or incomplete answers.
• —should not be used as the sole basis for legal, religious, or financial decisions.

Users should verify important information with authentic scholarly sources.

Training Data

The model was fine-tuned using an instruction dataset focused on Islamic finance.

The dataset contains question-answer style instruction examples covering major concepts of Islamic finance.

Training Procedure

Base Model

• —Qwen2.5-3B-Instruct

Method

• —QLoRA

Quantization

• —4-bit NF4
• —bitsandbytes

Parameter Efficient Fine Tuning

• —PEFT LoRA

Trainer

• —Hugging Face TRL SFTTrainer

Example

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-3B-Instruct"
)

model = PeftModel.from_pretrained(
    base_model,
    "zaidrahim162/qwen2.5-3b-islamic-finance-qlora"
)

tokenizer = AutoTokenizer.from_pretrained(
    "zaidrahim162/qwen2.5-3b-islamic-finance-qlora"
)

prompt = "Explain Murabaha."

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=256)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Citation

If you use this model in research, please cite this repository.

Acknowledgements

• —Alibaba Qwen Team
• —Hugging Face
• —PEFT
• —TRL
• —bitsandbytes