saiteki-kai/QA-DeBERTa-v3-large-threshold-smoothing

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

QA-DeBERTa-v3-large-threshold-smoothing

This model is a fine-tuned version of microsoft/deberta-v3-large on the saiteki-kai/Beavertails-it dataset. It achieves the following results on the evaluation set:

• —Loss: 0.0814
• —Accuracy: 0.6745
• —Macro F1: 0.6766
• —Macro Precision: 0.6733
• —Macro Recall: 0.6886
• —Micro F1: 0.7547
• —Micro Precision: 0.7403
• —Micro Recall: 0.7697
• —Flagged/accuracy: 0.8571
• —Flagged/precision: 0.8625
• —Flagged/recall: 0.8842
• —Flagged/f1: 0.8732
• —Flagged/aucpr: 0.9056
• —Flagged/fpr: 0.1769
• —Animal Abuse/accuracy: 0.9950
• —Animal Abuse/precision: 0.8
• —Animal Abuse/recall: 0.7558
• —Animal Abuse/f1: 0.7773
• —Animal Abuse/fpr: 0.0022
• —Animal Abuse/threshold: 0.3858
• —Child Abuse/accuracy: 0.9965
• —Child Abuse/precision: 0.6911
• —Child Abuse/recall: 0.6517
• —Child Abuse/f1: 0.6708
• —Child Abuse/fpr: 0.0016
• —Child Abuse/threshold: 0.4537
• —Controversial Topics,politics/accuracy: 0.9661
• —Controversial Topics,politics/precision: 0.4602
• —Controversial Topics,politics/recall: 0.6091
• —Controversial Topics,politics/f1: 0.5243
• —Controversial Topics,politics/fpr: 0.0226
• —Controversial Topics,politics/threshold: 0.2524
• —Discrimination,stereotype,injustice/accuracy: 0.9547
• —Discrimination,stereotype,injustice/precision: 0.7064
• —Discrimination,stereotype,injustice/recall: 0.7360
• —Discrimination,stereotype,injustice/f1: 0.7209
• —Discrimination,stereotype,injustice/fpr: 0.0264
• —Discrimination,stereotype,injustice/threshold: 0.3381
• —Drug Abuse,weapons,banned Substance/accuracy: 0.9736
• —Drug Abuse,weapons,banned Substance/precision: 0.7409
• —Drug Abuse,weapons,banned Substance/recall: 0.8160
• —Drug Abuse,weapons,banned Substance/f1: 0.7767
• —Drug Abuse,weapons,banned Substance/fpr: 0.0170
• —Drug Abuse,weapons,banned Substance/threshold: 0.5554
• —Financial Crime,property Crime,theft/accuracy: 0.9595
• —Financial Crime,property Crime,theft/precision: 0.7607
• —Financial Crime,property Crime,theft/recall: 0.8523
• —Financial Crime,property Crime,theft/f1: 0.8039
• —Financial Crime,property Crime,theft/fpr: 0.0289
• —Financial Crime,property Crime,theft/threshold: 0.4458
• —Hate Speech,offensive Language/accuracy: 0.9495
• —Hate Speech,offensive Language/precision: 0.7431
• —Hate Speech,offensive Language/recall: 0.6668
• —Hate Speech,offensive Language/f1: 0.7029
• —Hate Speech,offensive Language/fpr: 0.0227
• —Hate Speech,offensive Language/threshold: 0.2720
• —Misinformation Regarding Ethics,laws And Safety/accuracy: 0.9836
• —Misinformation Regarding Ethics,laws And Safety/precision: 0.2764
• —Misinformation Regarding Ethics,laws And Safety/recall: 0.2148
• —Misinformation Regarding Ethics,laws And Safety/f1: 0.2417
• —Misinformation Regarding Ethics,laws And Safety/fpr: 0.0069
• —Misinformation Regarding Ethics,laws And Safety/threshold: 0.2018
• —Non Violent Unethical Behavior/accuracy: 0.8811
• —Non Violent Unethical Behavior/precision: 0.7037
• —Non Violent Unethical Behavior/recall: 0.6941
• —Non Violent Unethical Behavior/f1: 0.6989
• —Non Violent Unethical Behavior/fpr: 0.0725
• —Non Violent Unethical Behavior/threshold: 0.3513
• —Privacy Violation/accuracy: 0.9813
• —Privacy Violation/precision: 0.7974
• —Privacy Violation/recall: 0.8321
• —Privacy Violation/f1: 0.8144
• —Privacy Violation/fpr: 0.0110
• —Privacy Violation/threshold: 0.4893
• —Self Harm/accuracy: 0.9970
• —Self Harm/precision: 0.8859
• —Self Harm/recall: 0.6439
• —Self Harm/f1: 0.7458
• —Self Harm/fpr: 0.0006
• —Self Harm/threshold: 0.7746
• —Sexually Explicit,adult Content/accuracy: 0.9835
• —Sexually Explicit,adult Content/precision: 0.6369
• —Sexually Explicit,adult Content/recall: 0.7332
• —Sexually Explicit,adult Content/f1: 0.6817
• —Sexually Explicit,adult Content/fpr: 0.0103
• —Sexually Explicit,adult Content/threshold: 0.4321
• —Terrorism,organized Crime/accuracy: 0.9895
• —Terrorism,organized Crime/precision: 0.3898
• —Terrorism,organized Crime/recall: 0.5551
• —Terrorism,organized Crime/f1: 0.4580
• —Terrorism,organized Crime/fpr: 0.0070
• —Terrorism,organized Crime/threshold: 0.2018
• —Violence,aiding And Abetting,incitement/accuracy: 0.9211
• —Violence,aiding And Abetting,incitement/precision: 0.8333
• —Violence,aiding And Abetting,incitement/recall: 0.8792
• —Violence,aiding And Abetting,incitement/f1: 0.8556
• —Violence,aiding And Abetting,incitement/fpr: 0.0638
• —Violence,aiding And Abetting,incitement/threshold: 0.4855

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• —learning_rate: 1e-05
• —trainbatchsize: 64
• —evalbatchsize: 128
• —seed: 42
• —optimizer: Use OptimizerNames.ADAMWTORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizerargs=No additional optimizer arguments
• —lrschedulertype: linear
• —lrschedulerwarmup_steps: 1000
• —num_epochs: 10
• —labelsmoothingfactor: 0.1

Training results

Framework versions

• —Transformers 4.57.1
• —Pytorch 2.7.1+cu118
• —Datasets 4.4.1
• —Tokenizers 0.22.1