AmirMohseni/tfidf-logreg-v3-legal-guidance-prefix-seed42

TF-IDF + logistic regression for prefix-aware legal-guidance detection

This lightweight baseline detects whether legal guidance is already present in a cumulative user-turn prefix. It is a research baseline, not a legal-advice system, and must not be used to decide whether a person has a valid claim.

Method

The model combines word (1,2)-gram and char_wb (3,5)-gram TF-IDF features with logistic regression. Prefixes before first_guidance_user_turn_id are negative; the onset prefix and all subsequent prefixes are positive. Each prefix is weighted by the inverse of the conversation's number of user turns, so every conversation has total training weight 1.0.

Both the regularization parameter and probability threshold were selected on the silver validation split by conversation-weighted prefix macro-F1, with weighted positive-class F1 as the tie-breaker.

Data and configuration

• —Dataset: AmirMohseni/WildChat-Legal-Classification-V3-Hierarchical
• —Requested revision: main (latest at execution)
• —Resolved revision: 489b87c18e571e8c7e9a3b1c2463b79e50cef120
• —Training: 1,632 conversations / 4,865 prefixes
• —Validation: 290 conversations / 912 prefixes
• —Selected C: 0.25
• —Selected threshold: 0.42
• —Seed: 42
• —W&B run: https://wandb.ai/rl-research-team/legal-guidance-prefix-v3/runs/fru3wk9m

First-guidance-turn annotations are silver and were not independently human-validated for this run. Onset metrics must not be described as gold.

Validation results

Silver onset analysis

Repository artifacts

• —tfidf_prefix_pipeline.joblib: fitted feature union, classifier, threshold, and provenance
• —prefix_validation_metrics.json: complete metrics, candidate results, and selected threshold curve
• —prefix_validation_predictions.csv: per-prefix probabilities and predictions
• —onset_validation_predictions.csv: conversation-level onset predictions
• —run_metadata.json: data, feature, software, and split provenance

Limitations

The labels and onset locations used here are silver. This is a sparse lexical baseline and may be brittle to paraphrases, spelling changes, domain shift, and jurisdiction-specific terminology. It has not yet been evaluated on the frozen human gold set. Gold onset evaluation requires manual review of first_guidance_user_turn_id.

Loading

Only load pickle/joblib files from sources you trust.

import joblib

artifact = joblib.load("tfidf_prefix_pipeline.joblib")
X = artifact["features"].transform(["User: example text"])
probability = artifact["classifier"].predict_proba(X)[:, 1]
prediction = (probability >= artifact["guidance_threshold"]).astype(int)