przvl/persuasive_essays_distilbert_cased

persuasiveessaysdistilbert_cased

Model description

This model is a fine-tuned version of distilbert-base-cased on the emnlp2017-claim-identification/persuasive_essays dataset. It achieves the following results on the evaluation set:

• —Loss: 0.4249
• —Accuracy: 0.8101
• —Macro F1: 0.7662
• —Claim F1: 0.665

Intended uses & limitations

Text classification for claims on full sentences. The model perfoms better at in-domain classification. Cross-domain classification is severely limited.

Training and evaluation data

Based on Stab and Gurevych (2017) persuasive essays corpus, preprocessed by Daxenberger et al. (2017).

Original dataset

• —docs: 402
• —tokens: 147,271
• —total instances: 7,116 (65 duplicates)
• —#claims: 2,108 (29.62%)

Trimmed datast used for training

• —total instances: 7051 (65 duplicates removed)
• —#claims: 2093 (29.68%)
• —train/test split: 80/20, stratified

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• —learning_rate: 2e-05
• —trainbatchsize: 16
• —evalbatchsize: 16
• —seed: 42
• —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• —lrschedulertype: linear
• —num_epochs: 2
• —mixedprecisiontraining: Native AMP

Training results

Framework versions

• —Transformers 4.37.2
• —Pytorch 2.2.0
• —Datasets 2.17.0
• —Tokenizers 0.15.2