Edric2412/Qwen3-8B-Hallucination-Detector-LoRA

Qwen3-8B Hallucination Detector (LoRA)

This model is a fine-tuned LoRA (Low-Rank Adaptation) adapter for Qwen3-8B, specifically trained to detect hallucinations in Large Language Model (LLM) responses.

Model Details

• —Base Model: Qwen/Qwen3-8B
• —Task: Text Classification / Hallucination Detection
• —Language: English
• —Adapter Architecture: QLoRA (Rank 16, Alpha 16, Dropout 0.05)
• —Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Usage (Inference via peft)

You can load this adapter on top of the base Qwen3-8B model using the Hugging Face peft and transformers libraries.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "Qwen/Qwen3-8B"
lora_model_id = "Edric2412/Qwen3-8B-Hallucination-Detector-LoRA"

# 1. Load Base Model and Tokenizer
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id, 
    device_map="auto",
    torch_dtype=torch.bfloat16
)

# 2. Load the LoRA adapter
model = PeftModel.from_pretrained(base_model, lora_model_id)

# 3. Format input prompt
prompt = """Instruct: Evaluate the following response for hallucination.
Question: What is the capital of France?
Response: The capital of France is Berlin.
Output:"""

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

# 4. Generate prediction
outputs = model.generate(**inputs, max_new_tokens=10)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
# Expected Output: HALLUCINATION

Dataset & Training Data

The model was fine-tuned using the [anniee001/halueval-judged](https://huggingface.co/datasets/anniee001/halueval-judged) dataset. Samples consist of (Instruction, Question, Response, Label) pairs.

The output space is constrained to three specific classes:

1. 1.HALLUCINATION (The response fabricates information or directly contradicts the context/facts)
2. 2.NO_HALLUCINATION (The response is grounded and correct)
3. 3.UNCERTAIN (The request is ambiguous, or the AI legitimately refused to answer safely)

Evaluation Results

The model was evaluated on a held-out test split of 2,000 samples, demonstrating strong capability in distinguishing grounded responses from hallucinations.

Overall Performance:

• —Accuracy: 83.65%
• —F1 Score (Macro): 0.8611

Classification Report

Framework versions

• —PEFT 0.12.0
• —Transformers 4.44.2
• —Pytorch 2.4.0+cu121
• —Datasets 3.0.0
• —Unsloth (for optimized training)