AmirMohseni/modernbert-large-v3-seeks-guidance-user-len4096-seed42

ModernBERT-large for full-conversation legal-guidance detection

This checkpoint predicts whether a conversation's user is seeking legal guidance. It is the first stage of the full-conversation ModernBERT-large cascade from the Legal Guidance in the Wild study. Input contains chronological user messages only; assistant messages are excluded. The paired topic model is modernbert-large-v3-primary-topic-user-len4096-seed42.

This is a research routing model, not a legal-advice system. It must not be used to decide whether a person has a valid claim or needs professional counsel.

Labels

• —False: no legal guidance sought
• —True: legal guidance sought

Data

• —Dataset: AmirMohseni/WildChat-Legal-Classification-V3-Hierarchical
• —Requested revision: main (latest at run time)
• —Train fingerprint: 6ba4c2696e889276
• —Validation fingerprint: 403fe118d76c8360
• —Train / validation conversations: 1,632 / 290
• —Input mode: chronological user messages only

Dataset access follows the linked repository's sharing settings. The fingerprints identify the exact loaded splits even if main later changes.

Training configuration

The checkpoint was selected by validation macro-F1, followed by guidance threshold selection. The latest complete run selected threshold 0.38.

Silver-validation results

The paired topic checkpoint obtains 0.716 accuracy and 0.574 macro-F1 on the 134 silver guidance-positive validation conversations. These development results are not final adjudicated-gold estimates.

Inference

from transformers import AutoModelForSequenceClassification, AutoTokenizer

repo = "AmirMohseni/modernbert-large-v3-seeks-guidance-user-len4096-seed42"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo)

# Serialize chronological user turns exactly as in the training preprocessing.
inputs = tokenizer(user_only_conversation, return_tensors="pt", truncation=True,
                   max_length=4096)
logits = model(**inputs).logits
probability = logits.softmax(-1)[0, model.config.label2id["True"]].item()
seeks_guidance = probability >= 0.38

The threshold 0.38 is the frozen validation-selected operating point and is the value used for every reported result. The repository's class labels alone do not encode it, so an argmax over the two logits will not reproduce the cascade predictions.

Limitations

The checkpoint was trained on English-language public LLM interaction logs with silver labels, one source, and one seed. Inputs beyond 4,096 tokens are truncated. The dataset is jurisdiction-agnostic, may contain sensitive content, and is not representative of all legal-help seekers. Calibration and accuracy may shift across platforms, populations, jurisdictions, and time. Human review is required for consequential use.

Citation

Please cite the accompanying Legal Guidance in the Wild: How Users Seek Legal Help in Real-World LLM Conversations manuscript when it becomes available.