Azizkhan22/qwen2.5-7b-hadith-quran-qa-lora

Qwen2.5-3B Islamic Context Guard LoRA

Model Description

This repository contains a QLoRA fine-tuned adapter for the base model Qwen/Qwen2.5-3B-Instruct.

The model is designed to act as a retrieval-grounded Islamic knowledge assistant that strictly answers only using the provided context. The objective of fine-tuning was to reduce hallucinations and ensure that responses remain faithful to the supplied Quran or Hadith text.

The model was trained to:

• —Answer only using the provided context
• —Avoid hallucinating Quran verses or Hadith
• —Ask for clarification if the question is vague
• —Politely decline if the answer is not present in the provided context

This model is intended for Retrieval-Augmented Generation (RAG) systems related to Islamic knowledge.

Base Model

Base model:

Qwen/Qwen2.5-3B-Instruct

Architecture: Transformer-based causal language model.

Fine-Tuning Method

Fine-tuning was performed using QLoRA (Quantized LoRA).

Configuration:

• —Fine-tuning method: QLoRA
• —LoRA Rank: 16
• —Adapter type: LoRA
• —Quantization: 4-bit
• —Training format: Chat / instruction style

QLoRA allows efficient training by updating only a small number of parameters while keeping the base model frozen.

Training Objective

The model was trained with the following constraint:

Only answer using the retrieved context.

Training behavior encourages the model to:

1. 1.Provide answers only from the context
2. 2.Ask for clarification when a question is vague
3. 3.Politely refuse if the context does not contain the answer
4. 4.Avoid generating verses or hadith not present in the input

Dataset Format

The dataset follows a chat-style message format.

Each training example includes:

• —A system instruction
• —A user prompt containing context
• —The assistant response

Example training sample:

{
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful Islamic knowledge assistant. You can only share information that has been retrieved for you. If no information is available, politely say you cannot answer. If a question is too vague, ask for clarification. Never make up hadiths or verses."
    },
    {
      "role": "user",
      "content": "Context: Surah 20: Ta-Ha (Taha), Verse 30\n\nVerse: Aaron, my brother.\n\nQuestion: Give me a verse about paradise."
    },
    {
      "role": "assistant",
      "content": "Aaron, my brother. (Surah 20: Ta-Ha, Verse 30)"
    }
  ]
}

The dataset includes examples that teach the model to:

• —Stay grounded to context
• —Reject unrelated questions
• —Avoid hallucination

Intended Use

This model is designed for:

• —Islamic knowledge assistants
• —Quran and Hadith RAG systems
• —Retrieval-grounded question answering
• —Research on hallucination reduction in language models

Example usage pipeline:

Retriever → Context → Qwen2.5-3B + LoRA Adapter → Answer

Loading the Model

Since this repository contains only the LoRA adapter, the base model must be loaded first.

Example:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-3B-Instruct"
)

model = PeftModel.from_pretrained(
    base_model,
    "Azizkhan22/qwen2.5-3b-islamic-context-lora"
)

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-3B-Instruct")

Limitations

• —The model relies completely on the provided context.
• —Poor retrieval may lead to incomplete answers.
• —It does not independently verify the authenticity of religious texts.
• —It should not replace consultation with qualified Islamic scholars.

Ethical Considerations

Generating religious content incorrectly can be harmful. This model was trained specifically to reduce hallucination when dealing with Quran and Hadith text by forcing responses to stay grounded in the provided context.

Users should still verify religious information with reliable scholarly sources.

License

This adapter follows the same license as the base model.

Base model license:

Qwen/Qwen2.5-3B-Instruct