Modelpublic

mehta7408/qlora-mistral-green-patent

sourceHugging Faceupdated 7mo agoView on Hugging Face

0likes13downloads

Model Card

QLoRA Fine-Tuned Mistral-7B for Green Patent Classification

A QLoRA adapter for Mistral-7B-Instruct-v0.3, fine-tuned to classify patent claims as green technology (Y02) or non-green.

Model Details

Parameter	Value
Base Model	`mistralai/Mistral-7B-Instruct-v0.3`
Method	QLoRA (4-bit NF4 quantization + LoRA adapters)
Total Parameters	7,289,966,592
Trainable Parameters	41,943,040 (0.58%)
Training Examples	5,000 patent claims
Epochs	1
Final Training Loss	0.9651
Training Runtime	~104 minutes

Quantization Config

Setting	Value
Quantization	4-bit
Quant Type	NF4
Compute Dtype	float16
Double Quantization	Yes

Training Details

—Dataset: 5,000 patent claims from train_silver split of a 50k balanced green patent dataset
—Task: Given a patent claim, classify whether it describes green technology (Y02 classification)
—Prompt Format: Instruction-tuned format asking the model to classify and explain its reasoning
—Optimizer: AdamW with cosine learning rate schedule
—Max Sequence Length: 512 tokens

Training Loss Curve

Epoch	Loss
0.16	1.0607
0.32	0.9639
0.48	0.9558
0.64	0.9428
0.80	0.9361
0.96	0.9374

Intended Use

This adapter was used as the Advocate agent in a CrewAI Multi-Agent System for patent classification:

—Advocate (this model): Argues FOR green classification
—Skeptic (Groq Llama-3.1-8B): Argues AGAINST green classification
—Judge (Groq Llama-3.1-8B): Weighs both arguments and decides

How to Load

python

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel

# Quantization config (must match training)
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-Instruct-v0.3",
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
)

# Load QLoRA adapter
model = PeftModel.from_pretrained(base_model, "mehta7408/qlora-mistral-green-patent")
model.eval()

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("mehta7408/qlora-mistral-green-patent")
tokenizer.pad_token = tokenizer.eos_token

How to Run Inference

python

prompt = """[INST] You are a patent classification expert.
Classify whether this patent claim describes GREEN technology (Y02).
Answer with YES or NO and explain your reasoning.

PATENT CLAIM:
A method for reducing carbon dioxide emissions from a power plant
by capturing exhaust gases using a membrane-based separation system.

CLASSIFICATION: [/INST]"""

inputs = tokenizer(prompt, return_tensors="pt", truncation=True, max_length=512).to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=300,
        temperature=0.3,
        do_sample=True,
        top_p=0.9,
        pad_token_id=tokenizer.eos_token_id,
    )

new_tokens = output[0][inputs["input_ids"].shape[1]:]
response = tokenizer.decode(new_tokens, skip_special_tokens=True)
print(response)

Requirements

torch>=2.0
transformers>=4.36
peft>=0.7
bitsandbytes>=0.41
accelerate>=0.25

Limitations

—Fine-tuned on silver-labeled data (not manually verified), so predictions may contain noise
—Best used as one agent in a multi-agent ensemble rather than as a standalone classifier
—4-bit quantization required — cannot be loaded in full precision without the base model

License

Apache 2.0