Modelpublic

SatyamGhosh/qlora-legal-clause-extractor

sourceHugging Faceupdated 2mo agoView on Hugging Face

0likes8downloads

Model Card

Legal Clause Structured Extraction (QLoRA adapter)

LoRA adapter fine-tuned on top of Qwen/Qwen2.5-3B-Instruct (4-bit QLoRA) to convert real legal contract clauses into structured JSON: clause_type, risk_level, obligated_party, has_penalty, summary.

Data

Clause text is real, sourced from CUAD (Contract Understanding Atticus Dataset, Hendrycks et al., NeurIPS 2021, CC-BY-4.0) -- 333 real clauses sampled across CUAD's original 41 categories. Structured labels (risk level, obligated party, penalty flag, summary) were generated with an LLM-assisted labeling pipeline (llama-3.1-8b-instant) and spot-checked by hand.

Results (held-out validation set, n=49)

Metric	Baseline (zero-shot)	Fine-tuned (this adapter)
JSON validity	61.2%	100.0%
Field accuracy	33.9%	66.5%

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

bnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                                 bnb_4bit_compute_dtype=torch.bfloat16)
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-3B-Instruct", quantization_config=bnb_config, device_map="auto")
model = PeftModel.from_pretrained(base, "SatyamGhosh/qlora-legal-clause-extractor")
tokenizer = AutoTokenizer.from_pretrained("SatyamGhosh/qlora-legal-clause-extractor")