bugBug04S/legal-embed-modernbert-v2

legal-embed-modernbert-v2

Embedding model for legal retrieval: matches plain-English legal questions to the passages that answer them, and clause-type descriptions to the contract clauses that match. Fine-tuned from nomic-ai/modernbert-embed-base.

Use it for: legal Q&A search, contract clause lookup, RAG over legal corpora. Supports Matryoshka embeddings — truncate to 512/256/128/64 dims for cheaper search at modest quality cost.

v2 changes over [v1](https://hf.co/bugBug04S/legal-embed-modernbert-v1): hard-negative mining, contract-clause training data, Australian legal QA, and a general-domain slice to limit domain drift.

Evaluation

All three models evaluated under identical settings (max_seq_length=256, nomic prefixes applied). Best per column in bold.

Legal Q&A — 1,000 held-out Law StackExchange pairs, never seen in training, retrieval among 1,000 candidates:

Contractual Clause Retrieval (isaacus) — 45 clause types, 90 clauses:

LegalBench Consumer Contracts QA (mteb):

How to read these results

• —Clause retrieval: the large gain reflects training on CUAD clause-type data. The evaluation texts are from a different source, but the task format (clause-type query → clause text) matches the training data, so this is not a pure zero-shot result. Judge it as "trained for this task and does it well," not "generalises to unseen tasks."
• —Consumer contracts QA: v2 is marginally below v1 and level with the base. This benchmark asks natural questions about contract content rather than matching clause types, and v2's broader training mix traded a little of this for the clause-retrieval gain. If consumer-contract QA is your primary use case, evaluate v1 as well.
• —Numbers here are not comparable to those on the v1 model card, which used a different sequence-length setting for the baseline.

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("bugBug04S/legal-embed-modernbert-v2")

query = "search_query: can the client terminate early without cause?"
docs = [
    "search_document: Either party may terminate this Agreement upon thirty (30) days prior written notice.",
    "search_document: The Licensee shall indemnify and hold harmless the Licensor against all claims.",
]
q = model.encode(query, normalize_embeddings=True)
d = model.encode(docs, normalize_embeddings=True)
print(q @ d.T)  # higher = more relevant

Required: prepend search_query: to queries and search_document: to passages. The model is trained with these prefixes and degrades noticeably without them.

Training

• —Base: nomic-ai/modernbert-embed-base (149M params)
• —24,413 triplets (anchor, positive, mined hard negative):
• —~16K Law StackExchange Q&A pairs (CC-BY-SA 4.0)
• —~2.1K Australian legal QA (isaacus)
• —~2.4K CUAD contract clauses, capped at 60 per clause type (CC-BY 4.0)
• —4K general-domain pairs (Natural Questions) to limit domain drift
• —Hard negatives mined with v1; mean positive/negative similarity gap 0.15
• —Loss: MatryoshkaLoss(CachedMultipleNegativesRankingLoss), dims 768/512/256/128/64
• —1 epoch, batch 64, lr 2e-5, fp16, maxseqlength 256, NO_DUPLICATES batch sampler
• —Single T4 GPU, ~37 minutes

Limitations

• —Trained and evaluated at 256 tokens; long contracts should be chunked.
• —Predominantly US/UK/Australian sources. Other jurisdictions are underrepresented.
• —Law StackExchange answers are community-written and not authoritative.
• —Retrieval only — this model does not generate legal advice, and its output is not a substitute for a qualified lawyer.