Modelpublic

Yeji-Seong/distilbert-base-uncased-tokenclassification_lora

sourceHugging Faceapache-2.0updated 3y agoView on Hugging Face

0likes44downloads

Model Card

distilbert-base-uncased-tokenclassification_lora

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

—Loss: 0.2286
—Precision: 0.6655
—Recall: 0.4474
—F1: 0.5351
—Accuracy: 0.9493

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 2e-05
—trainbatchsize: 16
—evalbatchsize: 16
—seed: 42
—optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
—lrschedulertype: linear
—num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	1.0	213	0.4882	0.0	0.0	0.0	0.9205
No log	2.0	426	0.4615	0.0	0.0	0.0	0.9205
0.9185	3.0	639	0.4220	0.0	0.0	0.0	0.9205
0.9185	4.0	852	0.3565	0.0	0.0	0.0	0.9205
0.25	5.0	1065	0.3219	0.25	0.0012	0.0024	0.9207
0.25	6.0	1278	0.3121	0.4737	0.0323	0.0605	0.9231
0.25	7.0	1491	0.3071	0.4783	0.0658	0.1157	0.9256
0.1979	8.0	1704	0.3015	0.4695	0.1196	0.1907	0.9290
0.1979	9.0	1917	0.2841	0.4871	0.2033	0.2869	0.9342
0.1775	10.0	2130	0.2823	0.4932	0.2177	0.3021	0.9349
0.1775	11.0	2343	0.2729	0.5090	0.2703	0.3531	0.9374
0.1691	12.0	2556	0.2731	0.5273	0.2775	0.3636	0.9382
0.1691	13.0	2769	0.2644	0.5660	0.3182	0.4074	0.9402
0.1691	14.0	2982	0.2648	0.6107	0.3134	0.4142	0.9402
0.1546	15.0	3195	0.2611	0.6388	0.3469	0.4496	0.9419
0.1546	16.0	3408	0.2570	0.6409	0.3565	0.4581	0.9431
0.1461	17.0	3621	0.2515	0.6541	0.3732	0.4752	0.9444
0.1461	18.0	3834	0.2461	0.6415	0.3959	0.4896	0.9456
0.1382	19.0	4047	0.2434	0.6452	0.4067	0.4989	0.9463
0.1382	20.0	4260	0.2464	0.6673	0.3983	0.4989	0.9457
0.1382	21.0	4473	0.2429	0.6767	0.4031	0.5052	0.9460
0.1324	22.0	4686	0.2411	0.684	0.4091	0.5120	0.9462
0.1324	23.0	4899	0.2336	0.6654	0.4306	0.5229	0.9475
0.129	24.0	5112	0.2411	0.6737	0.4175	0.5155	0.9469
0.129	25.0	5325	0.2385	0.6901	0.4234	0.5248	0.9473
0.1235	26.0	5538	0.2328	0.6843	0.4330	0.5304	0.9482
0.1235	27.0	5751	0.2343	0.6877	0.4294	0.5287	0.9481
0.1235	28.0	5964	0.2300	0.6649	0.4462	0.5340	0.9488
0.1195	29.0	6177	0.2323	0.6790	0.4378	0.5324	0.9483
0.1195	30.0	6390	0.2351	0.6869	0.4330	0.5312	0.9482
0.1179	31.0	6603	0.2329	0.6811	0.4342	0.5303	0.9482
0.1179	32.0	6816	0.2326	0.6779	0.4330	0.5285	0.9482
0.1156	33.0	7029	0.2326	0.6807	0.4258	0.5239	0.9481
0.1156	34.0	7242	0.2328	0.6870	0.4306	0.5294	0.9481
0.1156	35.0	7455	0.2327	0.6716	0.4354	0.5283	0.9484
0.114	36.0	7668	0.2290	0.6614	0.4486	0.5346	0.9492
0.114	37.0	7881	0.2275	0.6597	0.4522	0.5366	0.9495
0.1121	38.0	8094	0.2285	0.6643	0.4498	0.5364	0.9493
0.1121	39.0	8307	0.2275	0.6626	0.4533	0.5384	0.9495
0.1113	40.0	8520	0.2323	0.6784	0.4390	0.5330	0.9488
0.1113	41.0	8733	0.2289	0.6715	0.4450	0.5353	0.9491
0.1113	42.0	8946	0.2281	0.6696	0.4510	0.5390	0.9494
0.1111	43.0	9159	0.2284	0.6625	0.4486	0.5350	0.9493
0.1111	44.0	9372	0.2270	0.6591	0.4510	0.5355	0.9495
0.1077	45.0	9585	0.2291	0.6667	0.4474	0.5354	0.9493
0.1077	46.0	9798	0.2289	0.6691	0.4450	0.5345	0.9492
0.1089	47.0	10011	0.2272	0.6591	0.4510	0.5355	0.9495
0.1089	48.0	10224	0.2283	0.6661	0.4486	0.5361	0.9493
0.1089	49.0	10437	0.2286	0.6655	0.4474	0.5351	0.9493
0.1097	50.0	10650	0.2286	0.6655	0.4474	0.5351	0.9493

Framework versions

—PEFT 0.7.1
—Transformers 4.36.2
—Pytorch 2.0.0+cu117
—Datasets 2.16.1
—Tokenizers 0.15.0