alexchrander/patent-sberta-green-finetuned-mas

Patent Green Technology Classifier (PatentSBERTa Fine-tuned + MAS)

A binary text classifier for detecting green/sustainable technology patent claims, built on top of AI-Growth-Lab/PatentSBERTa. This is the Assignment 3 model, extending Assignment 2 by replacing the simple LLM labeling step with a three-agent debate system (MAS).

Model Description

This model was fine-tuned on a balanced dataset of 35,100 patent claims with gold-enhanced labels derived from a Multi-Agent System (MAS) debate pipeline followed by a Human-in-the-Loop (HITL) review. It classifies patent claims as either green technology (1) or not green technology (0).

Training Data

• —Base dataset: AI-Growth-Lab/patents_claims_1.5m_traim_test
• —Silver labels: Derived from CPC Y02* codes (25,000 green + 25,000 not green)
• —Gold labels: 100 examples labeled via MAS debate → Human HITL workflow
• —Dataset: alexchrander/patents-green-mas-dataset

Training Procedure

Active Learning + MAS + HITL workflow:

1. 1.Reused frozen PatentSBERTa baseline and uncertainty scores from Assignment 2
2. 2.Selected the same 100 most uncertain examples via uncertainty sampling
3. 3.Used a three-agent debate system to suggest labels:
4. 4.Advocate (Mistral-7B-Instruct-v0.2) — argues FOR green classification
5. 5.Skeptic (Qwen2.5-7B-Instruct) — argues AGAINST green classification
6. 6.Judge (Meta-Llama-3-8B-Instruct) — weighs both arguments and produces final label
7. 7.Human reviewer assigned final gold labels based on the full debate
8. 8.Fine-tuned PatentSBERTa on the gold-enhanced dataset

Hyperparameters:

• —maxseqlength: 256
• —epochs: 1
• —learning_rate: 2e-5
• —batch_size: 16

Results

Comparison across all model versions

MAS vs Simple LLM label quality

The MAS produced significantly more balanced and confident labels than the simple LLM approach, though both models achieved the same downstream F1 score of 0.81.

Video

https://panopto.aau.dk/Panopto/Pages/Viewer.aspx?id=5283748b-c71c-473c-89ec-b3f9016361f4