Modelpublic

domenicrosati/deberta-v3-large-survey-fluency-rater

sourceHugging Facemitupdated 3y agoView on Hugging Face

0likes4downloads

Model Card

deberta-v3-large-survey-fluency-rater

This model is a fine-tuned version of microsoft/deberta-v3-large on the None dataset. It achieves the following results on the evaluation set:

—Loss: 1.3349
—Krippendorff: 0.4934
—Spearman: 0.5298
—Absolute Agreement: 0.4862
—Agreement Within One: 0.8631

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 6e-06
—trainbatchsize: 16
—evalbatchsize: 16
—seed: 42
—optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
—lrschedulertype: linear
—lrschedulerwarmup_steps: 1000
—num_epochs: 20
—mixedprecisiontraining: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Krippendorff	Spearman	Absolute Agreement	Agreement Within One
No log	1.0	50	1.9418	-0.2790	-0.0817	0.0417	0.9306
No log	2.0	100	1.9563	-0.3968	-0.3236	0.0417	0.8889
No log	3.0	150	1.9360	-0.2790	-0.0817	0.0417	0.9306
No log	4.0	200	1.9826	-0.3968	-0.3236	0.0417	0.8889
No log	5.0	250	1.9908	-0.3665	-0.3871	0.0556	0.7917
No log	6.0	300	1.8144	-0.2888	-0.1075	0.2778	0.7222
No log	7.0	350	1.9314	-0.1378	-0.0729	0.2639	0.75
No log	8.0	400	2.0468	-0.3631	-0.3911	0.1389	0.75
No log	9.0	450	2.0021	-0.1594	-0.1099	0.25	0.7639
1.536	10.0	500	2.0445	-0.2759	-0.1253	0.2639	0.7222
1.536	11.0	550	1.9957	-0.1583	-0.1012	0.2361	0.7917
1.536	12.0	600	2.0293	-0.2232	-0.1460	0.25	0.7639
1.536	13.0	650	2.0920	-0.2296	-0.2063	0.1667	0.8056
1.536	14.0	700	2.2456	-0.2580	-0.3090	0.2083	0.75
1.536	15.0	750	2.2635	-0.1791	-0.1913	0.2778	0.75
1.536	16.0	800	2.4517	-0.1779	-0.2208	0.25	0.7639
1.536	17.0	850	2.5741	-0.2354	-0.3037	0.2361	0.7361
1.536	18.0	900	2.6429	-0.1990	-0.2052	0.2361	0.7639
1.536	19.0	950	2.8684	-0.2073	-0.2513	0.1944	0.7639
0.7381	20.0	1000	2.9184	-0.2585	-0.2424	0.2639	0.7361

Framework versions

—Transformers 4.26.0
—Pytorch 1.13.1
—Datasets 2.10.1
—Tokenizers 0.12.1