Modelpublic

genki10/BERT_V8_sp10_lw20_ex200_lo100_k1_k1_fold0

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face

0likes4downloads

Model Card

BERTV8sp10lw20ex200lo100k1k1fold0

This model is a fine-tuned version of bert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

—Loss: 0.7692
—Qwk: 0.4596
—Mse: 0.7692
—Rmse: 0.8770

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 2e-05
—trainbatchsize: 64
—evalbatchsize: 64
—seed: 42
—optimizer: Use adamwtorch with betas=(0.9,0.999) and epsilon=1e-08 and optimizerargs=No additional optimizer arguments
—lrschedulertype: linear
—num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse	Rmse
No log	1.0	1	12.3815	0.0	12.3815	3.5187
No log	2.0	2	11.5639	0.0094	11.5639	3.4006
No log	3.0	3	11.2668	0.0036	11.2668	3.3566
No log	4.0	4	11.0321	0.0	11.0321	3.3215
No log	5.0	5	10.7933	0.0	10.7933	3.2853
No log	6.0	6	10.5285	0.0	10.5285	3.2448
No log	7.0	7	10.2196	0.0	10.2196	3.1968
No log	8.0	8	9.8434	0.0	9.8434	3.1374
No log	9.0	9	9.3796	0.0	9.3796	3.0626
No log	10.0	10	8.8148	0.0	8.8148	2.9690
No log	11.0	11	8.1611	0.0	8.1611	2.8568
No log	12.0	12	7.4566	0.0	7.4566	2.7307
No log	13.0	13	6.7440	0.0	6.7440	2.5969
No log	14.0	14	6.0365	0.0338	6.0365	2.4569
No log	15.0	15	5.3273	0.0170	5.3273	2.3081
No log	16.0	16	4.7094	0.0115	4.7094	2.1701
No log	17.0	17	4.2652	0.0077	4.2652	2.0652
No log	18.0	18	3.9036	0.0	3.9036	1.9758
No log	19.0	19	3.5500	0.0	3.5500	1.8841
No log	20.0	20	3.1901	0.0	3.1901	1.7861
No log	21.0	21	2.8448	0.0	2.8448	1.6867
No log	22.0	22	2.5363	0.0295	2.5363	1.5926
No log	23.0	23	2.2785	0.0932	2.2785	1.5095
No log	24.0	24	2.0591	0.0344	2.0591	1.4350
No log	25.0	25	1.8700	0.0316	1.8700	1.3675
No log	26.0	26	1.6993	0.0316	1.6993	1.3036
No log	27.0	27	1.5573	0.0316	1.5573	1.2479
No log	28.0	28	1.4071	0.0316	1.4071	1.1862
No log	29.0	29	1.2609	0.0316	1.2609	1.1229
No log	30.0	30	1.1422	0.0316	1.1422	1.0688
No log	31.0	31	1.0446	0.0316	1.0446	1.0221
No log	32.0	32	0.9618	0.0316	0.9618	0.9807
No log	33.0	33	0.8922	0.0667	0.8922	0.9446
No log	34.0	34	0.8252	0.3667	0.8252	0.9084
No log	35.0	35	0.7798	0.3411	0.7798	0.8830
No log	36.0	36	0.7345	0.3436	0.7345	0.8571
No log	37.0	37	0.7097	0.3359	0.7097	0.8425
No log	38.0	38	0.7227	0.4004	0.7227	0.8501
No log	39.0	39	0.7474	0.4207	0.7474	0.8645
No log	40.0	40	0.7645	0.4519	0.7645	0.8743
No log	41.0	41	0.7539	0.4422	0.7539	0.8683
No log	42.0	42	0.7516	0.4607	0.7516	0.8669
No log	43.0	43	0.8201	0.4797	0.8201	0.9056
No log	44.0	44	0.8247	0.5070	0.8247	0.9081
No log	45.0	45	0.7636	0.5330	0.7636	0.8738
No log	46.0	46	0.7374	0.5149	0.7374	0.8587
No log	47.0	47	0.7121	0.5180	0.7121	0.8438
No log	48.0	48	0.6470	0.4922	0.6470	0.8044
No log	49.0	49	0.6276	0.4802	0.6276	0.7922
No log	50.0	50	0.6433	0.4778	0.6433	0.8021
No log	51.0	51	0.6878	0.5041	0.6878	0.8293
No log	52.0	52	0.7077	0.4917	0.7077	0.8412
No log	53.0	53	0.6573	0.4913	0.6573	0.8108
No log	54.0	54	0.6137	0.5180	0.6137	0.7834
No log	55.0	55	0.6281	0.4919	0.6281	0.7925
No log	56.0	56	0.6882	0.4839	0.6882	0.8296
No log	57.0	57	0.6946	0.4775	0.6946	0.8334
No log	58.0	58	0.6640	0.4804	0.6640	0.8149
No log	59.0	59	0.6554	0.4887	0.6554	0.8095
No log	60.0	60	0.6936	0.4701	0.6936	0.8328
No log	61.0	61	0.7869	0.4376	0.7869	0.8871
No log	62.0	62	0.8185	0.4233	0.8185	0.9047
No log	63.0	63	0.8026	0.4289	0.8026	0.8959
No log	64.0	64	0.8232	0.4226	0.8232	0.9073
No log	65.0	65	0.8002	0.4364	0.8002	0.8946
No log	66.0	66	0.7713	0.4435	0.7713	0.8782
No log	67.0	67	0.7654	0.4509	0.7654	0.8749
No log	68.0	68	0.7678	0.4560	0.7678	0.8762
No log	69.0	69	0.7967	0.4298	0.7967	0.8926
No log	70.0	70	0.8132	0.4189	0.8132	0.9018
No log	71.0	71	0.7879	0.4364	0.7879	0.8877
No log	72.0	72	0.7387	0.4611	0.7387	0.8595
No log	73.0	73	0.7249	0.4567	0.7249	0.8514
No log	74.0	74	0.7321	0.4597	0.7321	0.8556
No log	75.0	75	0.7692	0.4596	0.7692	0.8770

Framework versions

—Transformers 4.47.0
—Pytorch 2.5.1+cu121
—Datasets 3.3.1
—Tokenizers 0.21.0