wagesj45/multilabel-toxic-comment-classifier

Multilabel Toxic Comment Classifier

This model produces seven independent moderation scores for English comments: toxicity, severe_toxicity, obscene, threat, insult, identity_attack, and sexual_explicit. It is a fine-tune of `datalama/mmBERT-small`, a multilingual ModernBERT/mmBERT base model.

This is a multi-label classifier, not a mutually exclusive class classifier. Every output is a sigmoid score and a comment can score highly on multiple dimensions. toxicity is trained as its own target; it is not computed as an OR of the other labels.

Intended use

Use the scores as one input to a broader content-moderation workflow. Choose and validate a separate decision threshold for every label and review decisions in context. Do not use this model as the sole basis for high-impact decisions about people.

Usage

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_id = "wagesj45/multilabel-toxic-comment-classifier"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id).eval()

inputs = tokenizer(
    "Your text to score goes here.",
    return_tensors="pt",
    truncation=True,
    max_length=256,
)
with torch.no_grad():
    scores = torch.sigmoid(model(**inputs).logits[0]).tolist()

result = {
    model.config.id2label.get(index, model.config.id2label.get(str(index))): score
    for index, score in enumerate(scores)
}
print(result)

The model's saved label order is:

1. 1.toxicity
2. 2.severe_toxicity
3. 3.obscene
4. 4.threat
5. 5.insult
6. 6.identity_attack
7. 7.sexual_explicit

Scores are model outputs, not calibrated probabilities. Thresholds should be chosen for the target application and monitored after deployment.

Training

The model was fine-tuned with transformers on `google/civil_comments` and `Heliosoph/Jigsaw-Toxic-Comments`. Civil Comments supplies all seven continuous annotation-fraction targets. Jigsaw supplies binary targets for the first six dimensions; its unavailable sexual_explicit label is masked from loss rather than treated as negative.

The combined corpus was shuffled and split deterministically 90/10 with seed

1. 1.Training used three epochs, a learning rate of 2e-5, batch sizes 16/32, weight decay 0.01, and a 256-token input limit. The exported weights are the best checkpoint, selected by validation macro ROC-AUC at epoch 2.

Evaluation

Evaluation used the held-out split described above. Metrics threshold soft annotation targets at 0.5 for F1; ROC-AUC uses the continuous model scores.

These results are not a measure of performance on arbitrary production comments or languages outside the evaluation data.

Limitations and risks

• —The training and evaluation comments are English. The multilingual base model does not establish validated performance for non-English text.
• —Toxicity labels are subjective and noisy, and data may carry historical or cultural biases.
• —Profanity, reclaimed language, identity terms, quotations, discussions of abuse, and strongly worded criticism can cause false positives.
• —The model can miss implicit, coded, contextual, or adversarially written abuse.
• —Use application-specific thresholds, appeals, human review, and monitoring for real moderation systems.

Licensing and provenance

The fine-tuned model artifacts in this repository are released under the Apache License 2.0. The listed base model and both listed Hugging Face training datasets use Apache-2.0 and CC0-1.0 licensing, respectively; refer to their dataset cards for source attribution and terms. This repository does not redistribute training rows. Any future hard-negative data must be documented with its source and license before use in a released model.