saiteki-kai/QA-DeBERTa-v3-large-threshold-SEP-Focal

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

QA-DeBERTa-v3-large-threshold-SEP-Focal

This model is a fine-tuned version of microsoft/deberta-v3-large on the saiteki-kai/Beavertails-it dataset. It achieves the following results on the evaluation set:

• —Loss: 0.0054
• —Accuracy: 0.6685
• —Macro F1: 0.6757
• —Macro Precision: 0.6552
• —Macro Recall: 0.7071
• —Micro F1: 0.7526
• —Micro Precision: 0.7328
• —Micro Recall: 0.7735
• —Flagged/accuracy: 0.8540
• —Flagged/precision: 0.8528
• —Flagged/recall: 0.8915
• —Flagged/f1: 0.8717
• —Flagged/aucpr: 0.9024
• —Flagged/fpr: 0.1930
• —Animal Abuse/accuracy: 0.9949
• —Animal Abuse/precision: 0.7991
• —Animal Abuse/recall: 0.7456
• —Animal Abuse/f1: 0.7714
• —Animal Abuse/fpr: 0.0022
• —Animal Abuse/threshold: 0.4732
• —Child Abuse/accuracy: 0.9957
• —Child Abuse/precision: 0.5876
• —Child Abuse/recall: 0.7658
• —Child Abuse/f1: 0.6649
• —Child Abuse/fpr: 0.0030
• —Child Abuse/threshold: 0.3648
• —Controversial Topics,politics/accuracy: 0.9667
• —Controversial Topics,politics/precision: 0.4657
• —Controversial Topics,politics/recall: 0.5901
• —Controversial Topics,politics/f1: 0.5206
• —Controversial Topics,politics/fpr: 0.0214
• —Controversial Topics,politics/threshold: 0.3975
• —Discrimination,stereotype,injustice/accuracy: 0.9535
• —Discrimination,stereotype,injustice/precision: 0.6881
• —Discrimination,stereotype,injustice/recall: 0.7600
• —Discrimination,stereotype,injustice/f1: 0.7223
• —Discrimination,stereotype,injustice/fpr: 0.0298
• —Discrimination,stereotype,injustice/threshold: 0.4603
• —Drug Abuse,weapons,banned Substance/accuracy: 0.9734
• —Drug Abuse,weapons,banned Substance/precision: 0.7418
• —Drug Abuse,weapons,banned Substance/recall: 0.8101
• —Drug Abuse,weapons,banned Substance/f1: 0.7744
• —Drug Abuse,weapons,banned Substance/fpr: 0.0168
• —Drug Abuse,weapons,banned Substance/threshold: 0.4683
• —Financial Crime,property Crime,theft/accuracy: 0.9604
• —Financial Crime,property Crime,theft/precision: 0.7794
• —Financial Crime,property Crime,theft/recall: 0.8274
• —Financial Crime,property Crime,theft/f1: 0.8027
• —Financial Crime,property Crime,theft/fpr: 0.0252
• —Financial Crime,property Crime,theft/threshold: 0.4847
• —Hate Speech,offensive Language/accuracy: 0.9491
• —Hate Speech,offensive Language/precision: 0.7290
• —Hate Speech,offensive Language/recall: 0.6865
• —Hate Speech,offensive Language/f1: 0.7071
• —Hate Speech,offensive Language/fpr: 0.0251
• —Hate Speech,offensive Language/threshold: 0.4671
• —Misinformation Regarding Ethics,laws And Safety/accuracy: 0.9765
• —Misinformation Regarding Ethics,laws And Safety/precision: 0.1988
• —Misinformation Regarding Ethics,laws And Safety/recall: 0.3078
• —Misinformation Regarding Ethics,laws And Safety/f1: 0.2415
• —Misinformation Regarding Ethics,laws And Safety/fpr: 0.0153
• —Misinformation Regarding Ethics,laws And Safety/threshold: 0.2975
• —Non Violent Unethical Behavior/accuracy: 0.8827
• —Non Violent Unethical Behavior/precision: 0.7085
• —Non Violent Unethical Behavior/recall: 0.6958
• —Non Violent Unethical Behavior/f1: 0.7021
• —Non Violent Unethical Behavior/fpr: 0.0710
• —Non Violent Unethical Behavior/threshold: 0.4644
• —Privacy Violation/accuracy: 0.9804
• —Privacy Violation/precision: 0.7747
• —Privacy Violation/recall: 0.8510
• —Privacy Violation/f1: 0.8111
• —Privacy Violation/fpr: 0.0128
• —Privacy Violation/threshold: 0.4632
• —Self Harm/accuracy: 0.9969
• —Self Harm/precision: 0.8478
• —Self Harm/recall: 0.6659
• —Self Harm/f1: 0.7459
• —Self Harm/fpr: 0.0008
• —Self Harm/threshold: 0.5402
• —Sexually Explicit,adult Content/accuracy: 0.9840
• —Sexually Explicit,adult Content/precision: 0.6506
• —Sexually Explicit,adult Content/recall: 0.7270
• —Sexually Explicit,adult Content/f1: 0.6867
• —Sexually Explicit,adult Content/fpr: 0.0096
• —Sexually Explicit,adult Content/threshold: 0.4235
• —Terrorism,organized Crime/accuracy: 0.9887
• —Terrorism,organized Crime/precision: 0.3686
• —Terrorism,organized Crime/recall: 0.5863
• —Terrorism,organized Crime/f1: 0.4526
• —Terrorism,organized Crime/fpr: 0.0081
• —Terrorism,organized Crime/threshold: 0.3923
• —Violence,aiding And Abetting,incitement/accuracy: 0.9213
• —Violence,aiding And Abetting,incitement/precision: 0.8337
• —Violence,aiding And Abetting,incitement/recall: 0.8795
• —Violence,aiding And Abetting,incitement/f1: 0.8560
• —Violence,aiding And Abetting,incitement/fpr: 0.0636
• —Violence,aiding And Abetting,incitement/threshold: 0.4710

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• —learning_rate: 1e-05
• —trainbatchsize: 64
• —evalbatchsize: 64
• —seed: 42
• —optimizer: Use OptimizerNames.ADAMWTORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizerargs=No additional optimizer arguments
• —lrschedulertype: cosine
• —lrschedulerwarmup_ratio: 0.03
• —num_epochs: 10

Training results

Framework versions

• —Transformers 4.57.1
• —Pytorch 2.7.1+cu118
• —Datasets 4.4.1
• —Tokenizers 0.22.1