tabularisai/ai-text-detection
Tabularis ModernBERT R1 — AI Text Detector
ModernBERT-base fine-tuned for binary AI-vs-human text classification. Trained on a unified ~11M-row corpus combining the RAID benchmark with six external AI-text datasets.
Real RAID leaderboard scores (PR #137)
These are the official numbers from the RAID benchmark CI run on the hidden test labels.
Beats candidate-D on most attack categories
The character-level attack losses (zero_width, homoglyph, whitespace) are closeable at inference time with NFKC normalization (see "Inference notes" below). Local pseudo-GT eval projects NFKC-normalized inference to push AUROC to ~0.993 and TPR@5% to ~0.996.
Training data
Labels manually verified per dataset before mixing.
Training recipe
- base model:
answerdotai/ModernBERT-base(149M params) - max sequence length: 512
- 1 epoch
- 4× H100 80GB, FSDP full-shard, BF16, TF32
- per-device batch 192, effective batch 768
- AdamW, lr 8e-5, 6% warmup, weight_decay 0.01
- class-weighted cross-entropy (inverse frequency): whuman=3.18, wAI=0.59
- label smoothing 0.005
- training time: 2h 25min total
Inference notes
Prediction is binary: score is the probability the text is AI-generated.
For maximum accuracy on noisy / adversarial inputs, apply NFKC normalization before scoring. Strips zero-width invisibles, fullwidth chars, ligatures; collapses whitespace. Projected leaderboard gain: AUROC +0.003, TPR@5% +0.014, TPR@1% +0.075 over raw inference.
import re, unicodedata
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
ZERO_WIDTH = re.compile(r"[-]")
WS = re.compile(r"\s+")
def normalize(text: str) -> str:
t = unicodedata.normalize("NFKC", text)
t = ZERO_WIDTH.sub("", t)
t = WS.sub(" ", t).strip()
return t
tok = AutoTokenizer.from_pretrained("tabularisai/ai-text-detection")
m = AutoModelForSequenceClassification.from_pretrained("tabularisai/ai-text-detection").eval().cuda()
@torch.no_grad()
def score(texts):
norm = [normalize(t) for t in texts]
enc = tok(norm, padding=True, truncation=True, max_length=512, return_tensors="pt").to("cuda")
with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
logits = m(**enc).logits
return torch.softmax(logits.float(), dim=-1)[:, 1].cpu().tolist()