CoolFace
Modelpublic

Kurapika993/qwen2.5-7b-responsible-ai-qlora

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face
2likes6downloads
Model Card

license: apache-2.0 basemodel: Qwen/Qwen2.5-7B-Instruct libraryname: peft tags:

  • —qlora
  • —lora
  • —peft
  • —responsible-ai
  • —toxicity
  • —bias
  • —safety
  • —qwen
  • —instruction-tuning
  • —sft
  • —trl
  • —bitsandbytes datasets:
  • —OxAISH-AL-LLM/wiki_toxic
  • —PKU-Alignment/BeaverTails
  • —Kurapika993/mini-responsible-ai-instruction-dataset ---

Qwen2.5-7B Responsible AI QLoRA Assistant

This repository contains a QLoRA adapter fine-tuned from Qwen/Qwen2.5-7B-Instruct for structured toxicity, bias, and safety-risk analysis. The finetuning was done using 3 different datasets to make the model robust.

Intended Use

As a safety model or gating model to audit toxicity, bias , safety-risks of generative ai outputs

Project Purpose

The goal of this project is to build a compact Responsible AI assistant that analyzes text or model responses and produces:

  • —Toxicity label
  • —Bias category
  • —Safety risk level
  • —Short explanations
  • —Safer rewrite

Output Format

The model is trained to answer in this format:

text
Toxicity label: ...
Bias category: ...
Safety risk: ...
Explanation: ...
Safer rewrite: ...

The repository includes:

text
responsible_ai_evaluation_outputs.json
training_config.json
loss_curve.png
responsible_ai_training_messages.jsonl

Example Usage (loading Model)

python
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel
import torch

base_model = "Qwen/Qwen2.5-7B-Instruct"
adapter = "Kurapika993/qwen2.5-7b-responsible-ai-qlora"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

tokenizer = AutoTokenizer.from_pretrained(adapter)

model = AutoModelForCausalLM.from_pretrained(
    base_model,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
)

model = PeftModel.from_pretrained(model, adapter)
model.eval()

Example Inference (after loading Model)

python
def generate_response(model, tokenizer, user_prompt, max_new_tokens=250):
    messages = [
        {
            "role": "system",
            "content": (
                "You are a responsible AI safety assistant. "
                "Analyze text for toxicity, bias, safety risk, and provide a safer rewrite. "
                "Always use the required structured output format."
            )
        },
        {
            "role": "user",
            "content": user_prompt
        }
    ]

    text = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True
    )

    inputs = tokenizer(
        text,
        return_tensors="pt"
    ).to(model.device)

    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=max_new_tokens,
            do_sample=True,
            temperature=0.3,
            top_p=0.9,
            repetition_penalty=1.05,
            pad_token_id=tokenizer.eos_token_id,
        )

    generated_tokens = outputs[0][inputs["input_ids"].shape[-1]:]
    response = tokenizer.decode(generated_tokens, skip_special_tokens=True)

    return response.strip()


test_text = "Women are naturally worse at engineering."

prompt = build_user_prompt(test_text, mode="text")

response = generate_response(
    model,
    tokenizer,
    prompt,
    max_new_tokens=250
)

print(response)
text
Expected Answer
--------------
Toxicity label: Toxic
Bias category: Gender
Safety risk: Medium
Explanation: The text stereotypes women as less capable in science or technology.
Safer rewrite: Engineering ability depends on skills, experience, and context, not gender.
python
test_text = "I hate this phone. The battery is terrible."

prompt = build_user_prompt(test_text, mode="text")

response = generate_response(
    model,
    tokenizer,
    prompt,
    max_new_tokens=250
)

print(response)
text
Expected Answer
--------------
Toxicity label: Non-toxic
Bias category: None detected
Safety risk: Low
Explanation: The text expresses frustration about an object, product, or service, not abuse toward a person or group.
Safer rewrite: I am frustrated with this software because the user interface is confusing.