saiteki-kai/QA-DeBERTa-v3-large-threshold-SEP

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

QA-DeBERTa-v3-large-threshold-SEP

This model is a fine-tuned version of microsoft/deberta-v3-large on the saiteki-kai/Beavertails-it dataset. It achieves the following results on the evaluation set:

• —Loss: 0.0807
• —Accuracy: 0.6740
• —Macro F1: 0.6751
• —Macro Precision: 0.6605
• —Macro Recall: 0.6974
• —Micro F1: 0.7525
• —Micro Precision: 0.7407
• —Micro Recall: 0.7648
• —Flagged/accuracy: 0.8552
• —Flagged/precision: 0.8607
• —Flagged/recall: 0.8828
• —Flagged/f1: 0.8716
• —Flagged/aucpr: 0.9043
• —Flagged/fpr: 0.1793
• —Animal Abuse/accuracy: 0.9947
• —Animal Abuse/precision: 0.7708
• —Animal Abuse/recall: 0.7674
• —Animal Abuse/f1: 0.7691
• —Animal Abuse/fpr: 0.0026
• —Animal Abuse/threshold: 0.4045
• —Child Abuse/accuracy: 0.9966
• —Child Abuse/precision: 0.7124
• —Child Abuse/recall: 0.6547
• —Child Abuse/f1: 0.6823
• —Child Abuse/fpr: 0.0015
• —Child Abuse/threshold: 0.2736
• —Controversial Topics,politics/accuracy: 0.9678
• —Controversial Topics,politics/precision: 0.4791
• —Controversial Topics,politics/recall: 0.5651
• —Controversial Topics,politics/f1: 0.5186
• —Controversial Topics,politics/fpr: 0.0194
• —Controversial Topics,politics/threshold: 0.2814
• —Discrimination,stereotype,injustice/accuracy: 0.9565
• —Discrimination,stereotype,injustice/precision: 0.7360
• —Discrimination,stereotype,injustice/recall: 0.7073
• —Discrimination,stereotype,injustice/f1: 0.7214
• —Discrimination,stereotype,injustice/fpr: 0.0219
• —Discrimination,stereotype,injustice/threshold: 0.4525
• —Drug Abuse,weapons,banned Substance/accuracy: 0.9733
• —Drug Abuse,weapons,banned Substance/precision: 0.7396
• —Drug Abuse,weapons,banned Substance/recall: 0.8122
• —Drug Abuse,weapons,banned Substance/f1: 0.7742
• —Drug Abuse,weapons,banned Substance/fpr: 0.0171
• —Drug Abuse,weapons,banned Substance/threshold: 0.4799
• —Financial Crime,property Crime,theft/accuracy: 0.9610
• —Financial Crime,property Crime,theft/precision: 0.7883
• —Financial Crime,property Crime,theft/recall: 0.8187
• —Financial Crime,property Crime,theft/f1: 0.8032
• —Financial Crime,property Crime,theft/fpr: 0.0237
• —Financial Crime,property Crime,theft/threshold: 0.4026
• —Hate Speech,offensive Language/accuracy: 0.9484
• —Hate Speech,offensive Language/precision: 0.7241
• —Hate Speech,offensive Language/recall: 0.6857
• —Hate Speech,offensive Language/f1: 0.7044
• —Hate Speech,offensive Language/fpr: 0.0257
• —Hate Speech,offensive Language/threshold: 0.3522
• —Misinformation Regarding Ethics,laws And Safety/accuracy: 0.9770
• —Misinformation Regarding Ethics,laws And Safety/precision: 0.1934
• —Misinformation Regarding Ethics,laws And Safety/recall: 0.2818
• —Misinformation Regarding Ethics,laws And Safety/f1: 0.2294
• —Misinformation Regarding Ethics,laws And Safety/fpr: 0.0145
• —Misinformation Regarding Ethics,laws And Safety/threshold: 0.1624
• —Non Violent Unethical Behavior/accuracy: 0.8818
• —Non Violent Unethical Behavior/precision: 0.7041
• —Non Violent Unethical Behavior/recall: 0.6986
• —Non Violent Unethical Behavior/f1: 0.7013
• —Non Violent Unethical Behavior/fpr: 0.0728
• —Non Violent Unethical Behavior/threshold: 0.3425
• —Privacy Violation/accuracy: 0.9811
• —Privacy Violation/precision: 0.7991
• —Privacy Violation/recall: 0.8247
• —Privacy Violation/f1: 0.8117
• —Privacy Violation/fpr: 0.0108
• —Privacy Violation/threshold: 0.5118
• —Self Harm/accuracy: 0.9967
• —Self Harm/precision: 0.7757
• —Self Harm/recall: 0.7171
• —Self Harm/f1: 0.7452
• —Self Harm/fpr: 0.0014
• —Self Harm/threshold: 0.3074
• —Sexually Explicit,adult Content/accuracy: 0.9829
• —Sexually Explicit,adult Content/precision: 0.6181
• —Sexually Explicit,adult Content/recall: 0.7630
• —Sexually Explicit,adult Content/f1: 0.6830
• —Sexually Explicit,adult Content/fpr: 0.0116
• —Sexually Explicit,adult Content/threshold: 0.4078
• —Terrorism,organized Crime/accuracy: 0.9883
• —Terrorism,organized Crime/precision: 0.3615
• —Terrorism,organized Crime/recall: 0.5967
• —Terrorism,organized Crime/f1: 0.4502
• —Terrorism,organized Crime/fpr: 0.0085
• —Terrorism,organized Crime/threshold: 0.1883
• —Violence,aiding And Abetting,incitement/accuracy: 0.9232
• —Violence,aiding And Abetting,incitement/precision: 0.8452
• —Violence,aiding And Abetting,incitement/recall: 0.8707
• —Violence,aiding And Abetting,incitement/f1: 0.8578
• —Violence,aiding And Abetting,incitement/fpr: 0.0578
• —Violence,aiding And Abetting,incitement/threshold: 0.5215

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• —learning_rate: 1e-05
• —trainbatchsize: 64
• —evalbatchsize: 64
• —seed: 42
• —optimizer: Use OptimizerNames.ADAMWTORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizerargs=No additional optimizer arguments
• —lrschedulertype: cosine
• —lrschedulerwarmup_ratio: 0.03
• —num_epochs: 10

Training results

Framework versions

• —Transformers 4.57.1
• —Pytorch 2.7.1+cu118
• —Datasets 4.4.1
• —Tokenizers 0.22.1