kokuren/jp-sns-offensiveness-estimator

jp-sns-offensiveness-estimator

Binary Japanese SNS offensiveness classifier.

• —Labels: NOT, OFFENSIVE
• —Operational REVIEW is a probability band, not a training class.
• —Selected model: student_tiny3l256
• —Base/init model: hotchpotch/japanese-reranker-tiny-v2
• —This pipeline does not initialize the encoder from scratch.
• —Dataset labels are secondary-audited/model-assisted, not human-gold.
• —Account-group split is used to reduce author leakage.

Test on the secondary-audited split

• —ROC-AUC: 0.9444
• —PR-AUC: 0.7783
• —Macro-F1: 0.8354
• —OFFENSIVE F1: 0.7086

These values measure agreement with the project's operational labels. They are not a universal real-world offensiveness benchmark.

Calibration

• —Temperature: 1.187770
• —Balanced threshold: 0.502

See metrics.json for the validation-derived review band.

Quantization

If present, onnx/model_int8.onnx is post-training dynamic INT8. TorchAO QAT is an optional experiment and is not used as the default artifact.

Intended use

Research, corpus ranking/triage, and assistive moderation experiments. Not intended for automatic punitive, legal, clinical, or employment decisions.