nutrientdocs/grounding-en

grounding-en

Does the document actually support this claim? grounding-en is a cross-encoder that scores whether a hypothesis (a number, date, or fact) is entailed by a premise drawn from a real document — a financial table, a filing, prose evidence.

It is the open, English member of Nutrient's grounding model family.

• —🎯 Try it: grounding-demo
• —🏆 Leaderboard: grounding-leaderboard
• —📊 Benchmark: grounding-benchmark

Results

On the held-out English grounding-benchmark (ROC-AUC), against the strongest open English NLI models:

grounding-en leads the field on the hard axis — number grounding .92 vs .48–.66 for general-purpose NLI models — while matching or beating them everywhere else. Full ranking on the leaderboard. The commercial sibling `grounding-multilingual` scores a touch higher again and covers 15+ languages.

Usage

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

m = "nutrientdocs/grounding-en"
tok = AutoTokenizer.from_pretrained(m)
model = AutoModelForSequenceClassification.from_pretrained(m).eval()

premise = "Revenue | 2023 | $4,213M\nRevenue | 2022 | $3,905M"
hypothesis = "2023 revenue was $4.2 billion."

enc = tok(premise, hypothesis, truncation=True, max_length=1024, return_tensors="pt")
with torch.no_grad():
    probs = torch.softmax(model(**enc).logits, dim=-1)[0]
p_support = probs[0].item()   # entailment is class index 0 (id2label = {0: entailment, 1: not_entailment})
print(f"grounded support = {p_support:.3f}")

An ONNX export is provided under `onnx/` for on-device / ONNX Runtime deployment.

Calibrating the score

Fine-tuning maximizes ranking (AUC), which tends to make the raw probability overconfident. For a score you can gate on ("0.9 means ~90% right"), apply temperature scaling — divide the logits by a fitted T before softmax. It's monotonic, so it leaves AUC/ranking untouched and only repairs the confidence values. On the serving distribution we fit T = 1.29 (ECE 0.028 → 0.009). Re-fit T on your distribution whenever your input pipeline changes.

Intended use & limits

• —Use it for: verifying extracted values against source documents, hallucination/citation checking, routing low-confidence extractions for review.
• —Limits: English only. The remaining ceiling is reasoning table-claim negatives and multi-step arithmetic. As a dedicated grounding model it trades a little general-NLI accuracy for grounding.

License & training data

Weights are Apache-2.0. Trained on a multi-corpus grounding set. The public, redistributable slice of the evaluation data is grounding-benchmark (CC-BY-SA-4.0); the full training set is not redistributed.

About the author

<a href="https://nutrient.io/"> <img src="https://avatars2.githubusercontent.com/u/1527679?v=3&s=200" height="80" /> </a>

This project is maintained and funded by Nutrient - The deterministic document infrastructure enterprises run their highest-stakes workflows on: replayable output, clear exceptions, and full audit trails on the messy, regulated documents where AI alone breaks.