saiteki-kai/QA-DeBERTa-v3-large-focal

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

QA-DeBERTa-v3-large-focal

This model is a fine-tuned version of microsoft/deberta-v3-large on the saiteki-kai/Beavertails-it dataset. It achieves the following results on the evaluation set:

• —Loss: 0.0166
• —Accuracy: 0.6855
• —Macro F1: 0.6318
• —Macro Precision: 0.6963
• —Macro Recall: 0.6278
• —Micro F1: 0.7545
• —Micro Precision: 0.7724
• —Micro Recall: 0.7373
• —Flagged/accuracy: 0.8545
• —Flagged/precision: 0.8830
• —Flagged/recall: 0.8514
• —Flagged/f1: 0.8669
• —Flagged/aucpr: 0.9085
• —Flagged/fpr: 0.1416
• —Animal Abuse/accuracy: 0.9945
• —Animal Abuse/precision: 0.7449
• —Animal Abuse/recall: 0.7936
• —Animal Abuse/f1: 0.7685
• —Animal Abuse/fpr: 0.0031
• —Animal Abuse/threshold: 0.5
• —Child Abuse/accuracy: 0.9958
• —Child Abuse/precision: 0.6010
• —Child Abuse/recall: 0.7417
• —Child Abuse/f1: 0.6640
• —Child Abuse/fpr: 0.0027
• —Child Abuse/threshold: 0.5
• —Controversial Topics,politics/accuracy: 0.9731
• —Controversial Topics,politics/precision: 0.6143
• —Controversial Topics,politics/recall: 0.3312
• —Controversial Topics,politics/f1: 0.4303
• —Controversial Topics,politics/fpr: 0.0066
• —Controversial Topics,politics/threshold: 0.5
• —Discrimination,stereotype,injustice/accuracy: 0.9552
• —Discrimination,stereotype,injustice/precision: 0.7141
• —Discrimination,stereotype,injustice/recall: 0.7272
• —Discrimination,stereotype,injustice/f1: 0.7206
• —Discrimination,stereotype,injustice/fpr: 0.0252
• —Discrimination,stereotype,injustice/threshold: 0.5
• —Drug Abuse,weapons,banned Substance/accuracy: 0.9710
• —Drug Abuse,weapons,banned Substance/precision: 0.6981
• —Drug Abuse,weapons,banned Substance/recall: 0.8535
• —Drug Abuse,weapons,banned Substance/f1: 0.7680
• —Drug Abuse,weapons,banned Substance/fpr: 0.0220
• —Drug Abuse,weapons,banned Substance/threshold: 0.5
• —Financial Crime,property Crime,theft/accuracy: 0.9600
• —Financial Crime,property Crime,theft/precision: 0.7710
• —Financial Crime,property Crime,theft/recall: 0.8385
• —Financial Crime,property Crime,theft/f1: 0.8033
• —Financial Crime,property Crime,theft/fpr: 0.0269
• —Financial Crime,property Crime,theft/threshold: 0.5
• —Hate Speech,offensive Language/accuracy: 0.9502
• —Hate Speech,offensive Language/precision: 0.7607
• —Hate Speech,offensive Language/recall: 0.6482
• —Hate Speech,offensive Language/f1: 0.7000
• —Hate Speech,offensive Language/fpr: 0.0201
• —Hate Speech,offensive Language/threshold: 0.5
• —Misinformation Regarding Ethics,laws And Safety/accuracy: 0.9873
• —Misinformation Regarding Ethics,laws And Safety/precision: 0.3621
• —Misinformation Regarding Ethics,laws And Safety/recall: 0.0575
• —Misinformation Regarding Ethics,laws And Safety/f1: 0.0992
• —Misinformation Regarding Ethics,laws And Safety/fpr: 0.0012
• —Misinformation Regarding Ethics,laws And Safety/threshold: 0.5
• —Non Violent Unethical Behavior/accuracy: 0.8885
• —Non Violent Unethical Behavior/precision: 0.7628
• —Non Violent Unethical Behavior/recall: 0.6367
• —Non Violent Unethical Behavior/f1: 0.6941
• —Non Violent Unethical Behavior/fpr: 0.0491
• —Non Violent Unethical Behavior/threshold: 0.5
• —Privacy Violation/accuracy: 0.9809
• —Privacy Violation/precision: 0.8031
• —Privacy Violation/recall: 0.8129
• —Privacy Violation/f1: 0.8080
• —Privacy Violation/fpr: 0.0103
• —Privacy Violation/threshold: 0.5
• —Self Harm/accuracy: 0.9965
• —Self Harm/precision: 0.7558
• —Self Harm/recall: 0.7171
• —Self Harm/f1: 0.7359
• —Self Harm/fpr: 0.0016
• —Self Harm/threshold: 0.5
• —Sexually Explicit,adult Content/accuracy: 0.9836
• —Sexually Explicit,adult Content/precision: 0.6512
• —Sexually Explicit,adult Content/recall: 0.6904
• —Sexually Explicit,adult Content/f1: 0.6702
• —Sexually Explicit,adult Content/fpr: 0.0091
• —Sexually Explicit,adult Content/threshold: 0.5
• —Terrorism,organized Crime/accuracy: 0.9923
• —Terrorism,organized Crime/precision: 0.6667
• —Terrorism,organized Crime/recall: 0.0707
• —Terrorism,organized Crime/f1: 0.1278
• —Terrorism,organized Crime/fpr: 0.0003
• —Terrorism,organized Crime/threshold: 0.5
• —Violence,aiding And Abetting,incitement/accuracy: 0.9221
• —Violence,aiding And Abetting,incitement/precision: 0.8422
• —Violence,aiding And Abetting,incitement/recall: 0.8701
• —Violence,aiding And Abetting,incitement/f1: 0.8559
• —Violence,aiding And Abetting,incitement/fpr: 0.0591
• —Violence,aiding And Abetting,incitement/threshold: 0.5

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• —learning_rate: 6e-06
• —trainbatchsize: 64
• —evalbatchsize: 512
• —seed: 42
• —optimizer: Use OptimizerNames.ADAMWTORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizerargs=No additional optimizer arguments
• —lrschedulertype: linear
• —lrschedulerwarmup_steps: 1000
• —num_epochs: 10

Training results

Framework versions

• —Transformers 4.57.3
• —Pytorch 2.7.1+cu118
• —Datasets 4.4.1
• —Tokenizers 0.22.1