Modelpublic

derogab/Sherlock-4B-QLoRA

sourceHugging Faceapache-2.0updated 17d agoView on Hugging Face

0likes32downloads

Model Card

Sherlock-4B-QLoRA

Work in progress: This adapter is still under active development.

QLoRA adapter for structured information extraction: (JSON schema + text) → JSON. Missing fields become null; unrelated text is ignored.

—Base model: `Qwen/Qwen3-4B-Instruct-2507`
—Training dataset: `derogab/Sherlock-Case-Files`
—Method: NF4 QLoRA (rank 16, alpha 32, dropout 0.05)
—Task: text-generation

Usage

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")
model = PeftModel.from_pretrained(base, "derogab/Sherlock-4B-QLoRA")
tokenizer = AutoTokenizer.from_pretrained("derogab/Sherlock-4B-QLoRA")

Benchmark

Sherlock is evaluated against the base model on structured extraction quality. Rates are percentages; Δ is in percentage points (higher is better). The 95% CI of Δ is a Newcombe score interval from the aggregate counts.

| Metric | Base | Sherlock | Δ (Sherlock − Base) | | --- | ---: | ---: | ---: | | Valid JSON | 100.0% | 100.0% | +0.0 pp | | Schema conformance | 100.0% | 100.0% | +0.0 pp | | Field accuracy | 95.2% | 99.1% | +3.9 pp | | Exact match | 78.0% | 94.0% | +16.0 pp |