Modelpublic

sangkm/go-emotions-fine-tuned-distilroberta

sourceHugging Faceapache-2.0updated 2y agoView on Hugging Face

0likes15downloads

Model Card

go-emotions-fine-tuned-distilroberta

This model is a fine-tuned version of distilbert/distilroberta-base on GoEmotions dataset. It achieves the following results on the evaluation set (threshold = 0.5):

—Loss: 0.0841
—Micro Precision: 0.6789
—Micro Recall: 0.5047
—Micro F1: 0.5790
—Macro Precision: 0.5559
—Macro Recall: 0.4000
—Macro F1: 0.4502
—Weighted Precision: 0.6538
—Weighted Recall: 0.5047
—Weighted F1: 0.5577
—Hamming Loss: 0.0308

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 5e-05
—trainbatchsize: 8
—evalbatchsize: 8
—seed: 42
—optimizer: Use OptimizerNames.ADAMWTORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizerargs=No additional optimizer arguments
—lrschedulertype: linear
—num_epochs: 3.0

Training results

Training Loss	Epoch	Step	Validation Loss	Micro Precision	Micro Recall	Micro F1	Macro Precision	Macro Recall	Macro F1	Weighted Precision	Weighted Recall	Weighted F1	Hamming Loss
0.1062	1.0	5427	0.0889	0.6956	0.4498	0.5464	0.5087	0.3111	0.3537	0.6246	0.4498	0.4936	0.0314
0.0828	2.0	10854	0.0834	0.7042	0.4798	0.5707	0.5874	0.3562	0.4108	0.6872	0.4798	0.5306	0.0303
0.0704	3.0	16281	0.0841	0.6789	0.5047	0.5790	0.5559	0.4000	0.4502	0.6538	0.5047	0.5577	0.0308

Test results

Class	Precision	Recall	F1-Score	Support
admiration	0.69	0.73	0.71	504
amusement	0.79	0.87	0.83	264
anger	0.58	0.41	0.48	198
annoyance	0.45	0.16	0.24	320
approval	0.58	0.34	0.43	351
caring	0.51	0.29	0.37	135
confusion	0.57	0.38	0.46	153
curiosity	0.50	0.46	0.48	284
desire	0.70	0.36	0.48	83
disappointment	0.60	0.19	0.28	151
disapproval	0.42	0.29	0.34	267
disgust	0.63	0.33	0.44	123
embarrassment	0.82	0.38	0.52	37
excitement	0.57	0.33	0.42	103
fear	0.71	0.64	0.68	78
gratitude	0.94	0.90	0.92	352
grief	0.00	0.00	0.00	6
joy	0.69	0.54	0.61	161
love	0.82	0.84	0.83	238
nervousness	0.67	0.17	0.28	23
optimism	0.63	0.48	0.55	186
pride	0.00	0.00	0.00	16
realization	0.54	0.13	0.21	145
relief	0.00	0.00	0.00	11
remorse	0.58	0.77	0.66	56
sadness	0.67	0.49	0.57	156
surprise	0.61	0.44	0.51	141
neutral	0.73	0.54	0.62	1787
micro avg	0.68	0.51	0.58	6329
macro avg	0.57	0.41	0.46	6329
weighted avg	0.66	0.51	0.56	6329
samples avg	0.56	0.53	0.54	6329

Framework versions

—Transformers 4.47.0
—Pytorch 2.3.1+cu121
—Datasets 2.20.0
—Tokenizers 0.21.0