Modelpublic

DarrenJiaImbue/ai-detection-demo-qwen_3_4b

sourceHugging Facecc-by-nc-sa-4.0updated 3mo agoView on Hugging Face

0likes9downloads

Model Card

ai-detection-demo — Qwen3-4B QLoRA AI-text-edit detector

A QLoRA'd `Qwen/Qwen3-4B` fine-tuned for AI-vs-human text classification. Matches state-of-the-art numbers on our 2026-vintage dataset reproduction.

Results on our test set (n=7,658)

Mode	Accuracy	Macro F1
`human_vs_ai`	0.998	0.998
`human_vs_rest`	0.955	0.949
`ai_vs_rest`	0.962	0.958
Ternary	0.917	0.916

human_vs_ai mode drops ai_edited rows (n=5,106 remaining).

Architecture

—Backbone: Qwen/Qwen3-4B (3.2B params), loaded in 4-bit NF4 (QLoRA)
—Adapter: LoRA rank-8, alpha=16, on all attention + MLP projections (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj)
—Head: NormedLinear = LayerNorm(hiddensize) → Linear(hiddensize, 4). Replaces the default score head; the LayerNorm keeps initial loss well-scaled. Saved via PEFT's modules_to_save=["score"] convention so the head weights ship inside adapter_model.safetensors.
—Read position: last non-pad token's hidden state
—Trainable params: ~16 M LoRA + ~12 K head. Total artifact ~78 MB.

Files

file	purpose
`adapter_model.safetensors`	LoRA A/B weights + score head
`adapter_config.json`	PEFT config (target_modules, r, alpha)
`tokenizer.json`, `tokenizer_config.json`	Qwen3 tokenizer
`chat_template.jinja`	Qwen3 chat template (unused at inference, kept for compatibility)
`trainer_state.json`	training metrics per step

Usage

Recommended: use the reproducible pipeline in `imbue-ai/ai-detection-demo` — it handles base-model loading, adapter application, and score-head reattachment automatically.

bash

git clone https://github.com/imbue-ai/ai-detection-demo
cd ai-detection-demo && pip install -r requirements.txt
python -m qwen_3_4b.inference    # scores our test split, writes predictions.jsonl

Manual loading, if you must:

python

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

# ai-detection-demo repo defines NormedLinear (LayerNorm + Linear).
from qwen_3_4b.train import NormedLinear

qcfg = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                          bnb_4bit_compute_dtype=torch.bfloat16)
base = AutoModelForSequenceClassification.from_pretrained(
    "Qwen/Qwen3-4B", num_labels=4, quantization_config=qcfg,
)
# swap in NormedLinear where the default score head goes
hidden = base.config.hidden_size
base.score = NormedLinear(hidden, 4, device=base.device, dtype=torch.bfloat16)
model = PeftModel.from_pretrained(base, "DarrenJiaImbue/ai-detection-demo-qwen_3_4b")
model.eval()

Training data

Trained on `DarrenJiaImbue/ai-detection-demo-dataset` — 75,316 / 3,234 / 7,658 train/val/test rows across 6 source domains and 3 generator models.

Config: 1 epoch, effective batch 24 (per-device 3 × 8 GPUs × accum 1), constant LR 3e-5, bf16 compute, min_words=20 filter (matches original v3 provenance). Full config.yaml in the demo repo.

Score decoder

At inference the model produces 4-class logits. The demo pipeline exposes one derived column per input row:

—qwen_3_4b_score — E[bucket] / (n_buckets - 1) ∈ [0, 1] (higher = more AI)

License

CC BY-NC-SA 4.0. Non-commercial research use only.