DataForGood/balance-tes-haters-classifier

Balance Tes Haters — Harassment Classifier

Binary classifier for French social media comments: harassment (1) vs benign (0).

Built for the Balance Tes Haters project, which collects and analyses cyberbullying reports from Instagram, TikTok, YouTube and Twitter.

Architecture

This is a two-component model:

The encoder is not fine-tuned — only the MLP head was trained. This keeps the classifier small and the encoder swappable.

Performance

Evaluated on a stratified held-out test set (15% of annotated French comments):

Comparison with other frozen-embedding approaches on the same test set:

Usage

from huggingface_hub import hf_hub_download
from sentence_transformers import SentenceTransformer
import joblib
import numpy as np

# Load components
clf = joblib.load(hf_hub_download(
    repo_id="DataForGood/balance-tes-haters-classifier",
    filename="harassment_arctic_mlp.joblib",
))
encoder = SentenceTransformer("Snowflake/snowflake-arctic-embed-l-v2.0")

def predict(text: str) -> int:
    """Returns 1 (harassment) or 0 (benign)."""
    X = encoder.encode([text], convert_to_numpy=True)
    return int(clf.predict(X)[0])

def predict_proba(text: str) -> float:
    """Returns harassment probability between 0 and 1."""
    X = encoder.encode([text], convert_to_numpy=True)
    return float(clf.predict_proba(X)[0, 1])

# Examples
predict("<Insert hateful french comment>")   # → 1
predict("super vidéo, continue comme ça")  # → 0

Training Data

• —Real annotations: French social media comments manually annotated via the Balance Tes Haters platform, covering 11 harassment categories (injure, menaces, doxxing, incitation à la haine, etc.)
• —Split: 70% train / 15% val / 15% test (stratified)
• —The MLP was trained on the real split only (no synthetic augmentation for this checkpoint)

Categories detected

The model collapses all harassment categories into a single binary label:

• —0 — Absence de cyberharcèlement
• —1 — Any of: Cyberharcèlement, Injure, Diffamation, Menaces, Doxxing, Incitation au suicide, Incitation à la haine, Cyberharcèlement à caractère sexuel, and others

Limitations

• —Trained exclusively on French comments — not suitable for other languages
• —Sarcasm and context-dependent harassment may be misclassified
• —F1 of ~0.69 means roughly 1 in 10 harassment comments is missed and 1 in 10 benign comments is flagged
• —Should be used as a triage tool, not a final decision system — human review recommended for borderline cases

Dependencies

pip install sentence-transformers scikit-learn huggingface_hub