CoolFace
Modelpublic

perfecXion/intentguard-legal

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face
0likes8downloads
Model Card

IntentGuard — Legal & Compliance

![License](https://opensource.org/licenses/Apache-2.0) ![Accuracy](#performance) ![Size](#model-details) ![Latency](#performance) ![Format](#model-details)

Production-ready vertical intent classifier for LLM chatbot guardrails. Classifies user messages as `allow`, `deny`, or `abstain` to keep legal chatbots on-topic and within professional boundaries.

Research Article | perfecXion.ai | Finance Model | Healthcare Model | Legal Model


IntentGuard Model Family

ModelVerticalAccuracyOff-Topic Pass RateLink
intentguard-financeFinancial Services99.6%0.00%perfecXion/intentguard-finance
intentguard-healthcareHealthcare & Clinical98.9%0.98%perfecXion/intentguard-healthcare
intentguard-legalLegal & Compliance97.9%0.50%This model

Overview

The Problem

Legal chatbots carry significant liability risk — unauthorized practice of law, confidentiality breaches, and jurisdictional issues demand strict topic boundaries. An LLM that happily answers questions about celebrity gossip or sports while branded as a legal assistant undermines professional credibility and trust.

The Solution

IntentGuard uses a tiny DeBERTa-v3-xsmall model (22M parameters, 2.5MB quantized) to classify user intent in <30ms on CPU with three-way classification:

  • Allow — On-topic legal query, pass to the LLM
  • Deny — Off-topic, block with a polite redirect
  • Abstain — Ambiguous, escalate to secondary classifier or human review

Performance

MetricValue
Overall Accuracy97.9%
Legitimate Block Rate0.00%
Off-Topic Pass Rate0.50%
p99 Latency (CPU)<30ms
Model Size (ONNX INT8)2.5MB
Base Parameters22M (DeBERTa-v3-xsmall)
Expected Calibration Error<0.03

Model Details

PropertyValue
ArchitectureDeBERTa-v3-xsmall (fine-tuned for 3-way classification)
FormatONNX (INT8 quantized)
Version1.0
VerticalLegal (Law & Compliance)
GPU RequiredNo — runs on CPU
PublisherperfecXion.ai

Core Topics (Allow)

Contracts, litigation, employment law, intellectual property, criminal law, family law, real estate law, immigration, corporate law, compliance, privacy law, civil rights, estate planning, bankruptcy

Hard Exclusions (Deny)

Sports, entertainment, cooking, gaming, celebrity gossip, fashion, travel/leisure, fiction writing, relationship advice


Usage

Python (ONNX Runtime)

python
import onnxruntime as ort
from transformers import AutoTokenizer
import numpy as np

tokenizer = AutoTokenizer.from_pretrained("perfecXion/intentguard-legal")
session = ort.InferenceSession("model.onnx")

text = "What are the requirements for filing a patent application?"
inputs = tokenizer(text, return_tensors="np", max_length=128, truncation=True, padding="max_length")

logits = session.run(None, {
    "input_ids": inputs["input_ids"],
    "attention_mask": inputs["attention_mask"]
})[0]

labels = ["allow", "deny", "abstain"]
prediction = labels[np.argmax(logits)]
confidence = float(np.max(np.exp(logits) / np.sum(np.exp(logits))))

print(f"Intent: {prediction} (confidence: {confidence:.3f})")
# Output: Intent: allow (confidence: 0.996)

Docker

bash
docker pull ghcr.io/perfecxion/intentguard:legal-1.0
docker run -p 8080:8080 ghcr.io/perfecxion/intentguard:legal-1.0

curl -X POST http://localhost:8080/v1/classify \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "What are my rights as a tenant?"}]}'

pip

bash
pip install intentguard

from intentguard import IntentGuard
guard = IntentGuard.load("legal")
result = guard.classify("What are the requirements for filing a patent?")
print(result)  # Intent(label='allow', confidence=0.996)

Example Classifications

User MessagePredictedConfidenceCorrect?
"What are the requirements for filing a patent?"allow0.996
"Can my landlord evict me without notice?"allow0.995
"Who won the Super Bowl?"deny0.999
"Tell me a joke"deny0.996
"Is GDPR compliance required for US companies?"allow0.989
"What's the best recipe for pasta?"deny0.998

Citation

bibtex
@misc{thornton2025intentguard,
  title={IntentGuard: A Production-Grade Vertical Intent Classifier for LLM Guardrails},
  author={Thornton, Scott},
  year={2025},
  publisher={perfecXion.ai},
  url={https://perfecxion.ai/articles/intentguard-vertical-intent-classifier-llm-guardrails.html},
  note={Model: https://huggingface.co/perfecXion/intentguard-legal}
}

Quality Metrics

MetricResult
Accuracy (Legal vertical)97.9%
Legitimate Block Rate0.00%
Off-Topic Pass Rate0.50%
Expected Calibration Error<0.03
ONNX INT8 QuantizationValidated
CPU Inference (p99)<30ms

License

Apache 2.0


Links