saiteki-kai/QA-DeBERTa-v3-large-diff-binary

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

QA-DeBERTa-v3-large-diff-binary

This model is a fine-tuned version of microsoft/deberta-v3-large on the saiteki-kai/Beavertails-it dataset. It achieves the following results on the evaluation set:

• —Loss: 0.0823
• —Accuracy: 0.6890
• —Macro F1: 0.6419
• —Macro Precision: 0.7045
• —Macro Recall: 0.6272
• —Micro F1: 0.7567
• —Micro Precision: 0.7754
• —Micro Recall: 0.7390
• —Flagged/accuracy: 0.8562
• —Flagged/precision: 0.8819
• —Flagged/recall: 0.8563
• —Flagged/f1: 0.8689
• —Flagged/aucpr: 0.9091
• —Flagged/fpr: 0.1439
• —Animal Abuse/accuracy: 0.9945
• —Animal Abuse/precision: 0.7337
• —Animal Abuse/recall: 0.8169
• —Animal Abuse/f1: 0.7730
• —Animal Abuse/fpr: 0.0034
• —Animal Abuse/threshold: 0.5
• —Child Abuse/accuracy: 0.9964
• —Child Abuse/precision: 0.7007
• —Child Abuse/recall: 0.6186
• —Child Abuse/f1: 0.6571
• —Child Abuse/fpr: 0.0015
• —Child Abuse/threshold: 0.5
• —Controversial Topics,politics/accuracy: 0.9715
• —Controversial Topics,politics/precision: 0.5467
• —Controversial Topics,politics/recall: 0.4099
• —Controversial Topics,politics/f1: 0.4685
• —Controversial Topics,politics/fpr: 0.0107
• —Controversial Topics,politics/threshold: 0.5
• —Discrimination,stereotype,injustice/accuracy: 0.9564
• —Discrimination,stereotype,injustice/precision: 0.7313
• —Discrimination,stereotype,injustice/recall: 0.7146
• —Discrimination,stereotype,injustice/f1: 0.7229
• —Discrimination,stereotype,injustice/fpr: 0.0227
• —Discrimination,stereotype,injustice/threshold: 0.5
• —Drug Abuse,weapons,banned Substance/accuracy: 0.9742
• —Drug Abuse,weapons,banned Substance/precision: 0.7637
• —Drug Abuse,weapons,banned Substance/recall: 0.7847
• —Drug Abuse,weapons,banned Substance/f1: 0.7741
• —Drug Abuse,weapons,banned Substance/fpr: 0.0145
• —Drug Abuse,weapons,banned Substance/threshold: 0.5
• —Financial Crime,property Crime,theft/accuracy: 0.9601
• —Financial Crime,property Crime,theft/precision: 0.7676
• —Financial Crime,property Crime,theft/recall: 0.8464
• —Financial Crime,property Crime,theft/f1: 0.8051
• —Financial Crime,property Crime,theft/fpr: 0.0276
• —Financial Crime,property Crime,theft/threshold: 0.5
• —Hate Speech,offensive Language/accuracy: 0.9506
• —Hate Speech,offensive Language/precision: 0.7660
• —Hate Speech,offensive Language/recall: 0.6462
• —Hate Speech,offensive Language/f1: 0.7010
• —Hate Speech,offensive Language/fpr: 0.0194
• —Hate Speech,offensive Language/threshold: 0.5
• —Misinformation Regarding Ethics,laws And Safety/accuracy: 0.9879
• —Misinformation Regarding Ethics,laws And Safety/precision: 0.5179
• —Misinformation Regarding Ethics,laws And Safety/recall: 0.0397
• —Misinformation Regarding Ethics,laws And Safety/f1: 0.0737
• —Misinformation Regarding Ethics,laws And Safety/fpr: 0.0005
• —Misinformation Regarding Ethics,laws And Safety/threshold: 0.5
• —Non Violent Unethical Behavior/accuracy: 0.8880
• —Non Violent Unethical Behavior/precision: 0.7571
• —Non Violent Unethical Behavior/recall: 0.6422
• —Non Violent Unethical Behavior/f1: 0.6950
• —Non Violent Unethical Behavior/fpr: 0.0511
• —Non Violent Unethical Behavior/threshold: 0.5
• —Privacy Violation/accuracy: 0.9809
• —Privacy Violation/precision: 0.7844
• —Privacy Violation/recall: 0.8439
• —Privacy Violation/f1: 0.8131
• —Privacy Violation/fpr: 0.0120
• —Privacy Violation/threshold: 0.5
• —Self Harm/accuracy: 0.9965
• —Self Harm/precision: 0.7672
• —Self Harm/recall: 0.7073
• —Self Harm/f1: 0.7360
• —Self Harm/fpr: 0.0015
• —Self Harm/threshold: 0.5
• —Sexually Explicit,adult Content/accuracy: 0.9843
• —Sexually Explicit,adult Content/precision: 0.6691
• —Sexually Explicit,adult Content/recall: 0.6876
• —Sexually Explicit,adult Content/f1: 0.6783
• —Sexually Explicit,adult Content/fpr: 0.0084
• —Sexually Explicit,adult Content/threshold: 0.5
• —Terrorism,organized Crime/accuracy: 0.9921
• —Terrorism,organized Crime/precision: 0.5180
• —Terrorism,organized Crime/recall: 0.1497
• —Terrorism,organized Crime/f1: 0.2323
• —Terrorism,organized Crime/fpr: 0.0011
• —Terrorism,organized Crime/threshold: 0.5
• —Violence,aiding And Abetting,incitement/accuracy: 0.9221
• —Violence,aiding And Abetting,incitement/precision: 0.8400
• —Violence,aiding And Abetting,incitement/recall: 0.8736
• —Violence,aiding And Abetting,incitement/f1: 0.8565
• —Violence,aiding And Abetting,incitement/fpr: 0.0603
• —Violence,aiding And Abetting,incitement/threshold: 0.5

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• —learning_rate: 6e-06
• —trainbatchsize: 64
• —evalbatchsize: 64
• —seed: 42
• —optimizer: Use OptimizerNames.ADAMWTORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizerargs=No additional optimizer arguments
• —lrschedulertype: linear
• —lrschedulerwarmup_steps: 1000
• —num_epochs: 10

Training results

Framework versions

• —Transformers 4.57.1
• —Pytorch 2.7.1+cu118
• —Datasets 4.4.1
• —Tokenizers 0.22.1