Modelpublic

genki10/BERT_V8_sp10_lw20_ex100_lo100_k5_k5_fold0

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face

0likes6downloads

Model Card

BERTV8sp10lw20ex100lo100k5k5fold0

This model is a fine-tuned version of bert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

—Loss: 0.6471
—Qwk: 0.4573
—Mse: 0.6471
—Rmse: 0.8044

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 2e-05
—trainbatchsize: 64
—evalbatchsize: 64
—seed: 42
—optimizer: Use adamwtorch with betas=(0.9,0.999) and epsilon=1e-08 and optimizerargs=No additional optimizer arguments
—lrschedulertype: linear
—num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse	Rmse
No log	1.0	3	10.3063	0.0038	10.3063	3.2104
No log	2.0	6	7.3815	0.0	7.3815	2.7169
No log	3.0	9	5.2573	0.0194	5.2573	2.2929
No log	4.0	12	3.6395	0.0115	3.6395	1.9078
No log	5.0	15	2.3628	0.1301	2.3628	1.5371
No log	6.0	18	1.6951	0.0316	1.6951	1.3020
No log	7.0	21	1.1591	0.0316	1.1591	1.0766
No log	8.0	24	0.8789	0.1368	0.8789	0.9375
No log	9.0	27	0.7282	0.2676	0.7282	0.8533
No log	10.0	30	0.6875	0.1899	0.6875	0.8292
No log	11.0	33	0.7266	0.4000	0.7266	0.8524
No log	12.0	36	0.5874	0.3939	0.5874	0.7664
No log	13.0	39	0.8066	0.3715	0.8066	0.8981
No log	14.0	42	0.6048	0.4960	0.6048	0.7777
No log	15.0	45	0.5560	0.4212	0.5560	0.7457
No log	16.0	48	0.6877	0.4693	0.6877	0.8293
No log	17.0	51	0.5188	0.4704	0.5188	0.7203
No log	18.0	54	0.5410	0.4647	0.5410	0.7355
No log	19.0	57	0.5809	0.4848	0.5809	0.7622
No log	20.0	60	0.6684	0.4512	0.6684	0.8176
No log	21.0	63	0.6386	0.4198	0.6386	0.7991
No log	22.0	66	0.7855	0.4206	0.7855	0.8863
No log	23.0	69	0.6920	0.4310	0.6920	0.8319
No log	24.0	72	0.7619	0.4470	0.7619	0.8728
No log	25.0	75	0.6527	0.4870	0.6527	0.8079
No log	26.0	78	0.6743	0.4956	0.6743	0.8212
No log	27.0	81	0.6443	0.4902	0.6443	0.8027
No log	28.0	84	0.6876	0.4775	0.6876	0.8292
No log	29.0	87	0.6528	0.4829	0.6528	0.8079
No log	30.0	90	0.6873	0.4771	0.6873	0.8290
No log	31.0	93	0.6348	0.4744	0.6348	0.7967
No log	32.0	96	0.6704	0.4654	0.6704	0.8188
No log	33.0	99	0.6415	0.4851	0.6415	0.8009
No log	34.0	102	0.6721	0.4600	0.6721	0.8198
No log	35.0	105	0.6549	0.4667	0.6549	0.8093
No log	36.0	108	0.6592	0.4644	0.6592	0.8119
No log	37.0	111	0.6640	0.4427	0.6640	0.8149
No log	38.0	114	0.7006	0.4537	0.7006	0.8370
No log	39.0	117	0.6730	0.4448	0.6730	0.8204
No log	40.0	120	0.6809	0.4408	0.6809	0.8252
No log	41.0	123	0.6488	0.4699	0.6488	0.8055
No log	42.0	126	0.6523	0.4780	0.6523	0.8076
No log	43.0	129	0.6365	0.4778	0.6365	0.7978
No log	44.0	132	0.6566	0.4993	0.6566	0.8103
No log	45.0	135	0.6356	0.4754	0.6356	0.7972
No log	46.0	138	0.6558	0.4756	0.6558	0.8098
No log	47.0	141	0.6577	0.4757	0.6577	0.8110
No log	48.0	144	0.6985	0.4521	0.6985	0.8358
No log	49.0	147	0.6669	0.4498	0.6669	0.8167
No log	50.0	150	0.7159	0.4604	0.7159	0.8461
No log	51.0	153	0.6938	0.4141	0.6938	0.8329
No log	52.0	156	0.7655	0.4126	0.7655	0.8749
No log	53.0	159	0.7226	0.4467	0.7226	0.8500
No log	54.0	162	0.7136	0.4106	0.7136	0.8448
No log	55.0	165	0.7639	0.4375	0.7639	0.8740
No log	56.0	168	0.6975	0.4559	0.6975	0.8352
No log	57.0	171	0.6829	0.4432	0.6829	0.8264
No log	58.0	174	0.6803	0.4715	0.6803	0.8248
No log	59.0	177	0.6656	0.4396	0.6656	0.8158
No log	60.0	180	0.6879	0.4421	0.6879	0.8294
No log	61.0	183	0.6731	0.4558	0.6731	0.8204
No log	62.0	186	0.6576	0.4587	0.6576	0.8109
No log	63.0	189	0.6724	0.4612	0.6724	0.8200
No log	64.0	192	0.6637	0.4431	0.6637	0.8147
No log	65.0	195	0.6565	0.4783	0.6565	0.8103
No log	66.0	198	0.7004	0.4493	0.7004	0.8369
No log	67.0	201	0.6485	0.4506	0.6485	0.8053
No log	68.0	204	0.6679	0.4479	0.6679	0.8173
No log	69.0	207	0.6626	0.4516	0.6626	0.8140
No log	70.0	210	0.6443	0.4535	0.6443	0.8027
No log	71.0	213	0.6669	0.4580	0.6669	0.8166
No log	72.0	216	0.6394	0.4650	0.6394	0.7997
No log	73.0	219	0.6415	0.4650	0.6415	0.8009
No log	74.0	222	0.6471	0.4573	0.6471	0.8044

Framework versions

—Transformers 4.47.0
—Pytorch 2.5.1+cu121
—Datasets 3.3.1
—Tokenizers 0.21.0