Symio-ai/legal-citation-verifier

Symio-ai/legal-citation-verifier

Model Description

Legal Citation Verifier validates legal citations against known case law databases. Given a citation string (e.g., "Smith v. Jones, 123 So. 3d 456 (Fla. 5th DCA 2020)"), the model classifies it as VALID, INVALID, PARTIALLY_VALID, or FABRICATED with a confidence score.

This model is purpose-built for the GLACIER legal production pipeline to catch hallucinated citations before they reach court filings.

Intended Use

• —Primary: Pre-filing citation validation in the GLACIER Stage 5 WDC audit
• —Secondary: Real-time citation checking during legal research and drafting
• —Integration: Called by bedrock-legal and mcp-troops during WDC validation panels

Task Type

text-classification -- Multi-class classification of citation validity

Base Model

microsoft/deberta-v3-large -- Selected for its strong NLI performance which transfers well to citation verification (matching citation text against factual records)

Training Data

Data Pipeline

1. 1.Extract all citations from CourtListener opinions via regex + NER
2. 2.Cross-reference each citation against the full database
3. 3.Generate negative examples using controlled LLM hallucination
4. 4.Label: VALID (exact match), PARTIALLY_VALID (close match, wrong page/date), INVALID (no match found), FABRICATED (structurally valid but nonexistent)

Benchmark Criteria (90%+ Target)

Critical threshold: A FABRICATED citation passing as VALID is a zero-tolerance failure. The model must err on the side of flagging uncertain citations for human review.

GLACIER Pipeline Integration

STAGE 2 (Research) --> citation-verifier checks all retrieved citations
STAGE 3 (WDC #1)  --> citation-verifier validates theory citations
STAGE 5 (WDC #2)  --> citation-verifier audits every citation in final draft

Integration point: Called via bedrock-legal verify_citation or directly through SageMaker endpoint. Returns structured JSON:

{
  "citation": "Smith v. Jones, 123 So. 3d 456 (Fla. 5th DCA 2020)",
  "classification": "VALID",
  "confidence": 0.97,
  "matched_record": "courtlistener:op-12345",
  "treatment": "positive_cited"
}

Training Configuration

• —Epochs: 5
• —Learning rate: 2e-5 with linear warmup
• —Batch size: 32
• —Max sequence length: 256
• —Optimizer: AdamW with weight decay 0.01
• —Hardware: AWS SageMaker ml.g5.2xlarge

Limitations

• —Trained primarily on US federal and state court citations (FL, MS focus)
• —May underperform on administrative law, tribal court, or international citations
• —Citation format variations not in training data may cause false positives
• —Does not verify the substantive holding of the cited case, only its existence

Version History