Modelpublic

MayBashendy/ArabicNewSplits7_B_usingALLEssays_FineTuningAraBERT_run1_AugV5_k12_task7_organization

sourceHugging Faceupdated 2y agoView on Hugging Face

0likes11downloads

Model Card

ArabicNewSplits7BusingALLEssaysFineTuningAraBERTrun1AugV5k12task7organization

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

—Loss: 0.4984
—Qwk: 0.5079
—Mse: 0.4984
—Rmse: 0.7060

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 2e-05
—trainbatchsize: 8
—evalbatchsize: 8
—seed: 42
—optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
—lrschedulertype: linear
—num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse	Rmse
No log	0.0333	2	2.6243	-0.0262	2.6243	1.6200
No log	0.0667	4	1.3389	0.0470	1.3389	1.1571
No log	0.1	6	1.1413	-0.1740	1.1413	1.0683
No log	0.1333	8	1.3181	-0.2491	1.3181	1.1481
No log	0.1667	10	1.2171	-0.0299	1.2171	1.1032
No log	0.2	12	1.2887	-0.1098	1.2887	1.1352
No log	0.2333	14	1.1974	-0.0274	1.1974	1.0942
No log	0.2667	16	1.1780	-0.1003	1.1780	1.0853
No log	0.3	18	1.2410	-0.0104	1.2410	1.1140
No log	0.3333	20	1.0315	-0.0646	1.0315	1.0156
No log	0.3667	22	1.0069	-0.0439	1.0069	1.0034
No log	0.4	24	1.0234	-0.0396	1.0234	1.0116
No log	0.4333	26	1.0276	-0.0453	1.0276	1.0137
No log	0.4667	28	1.0462	0.0826	1.0462	1.0228
No log	0.5	30	1.0651	0.0365	1.0651	1.0320
No log	0.5333	32	1.0800	0.0177	1.0800	1.0392
No log	0.5667	34	1.0043	0.1011	1.0043	1.0021
No log	0.6	36	0.9658	0.0888	0.9658	0.9827
No log	0.6333	38	1.0246	0.1867	1.0246	1.0122
No log	0.6667	40	0.9925	0.2244	0.9925	0.9962
No log	0.7	42	0.8481	0.1359	0.8481	0.9209
No log	0.7333	44	0.9519	-0.0200	0.9519	0.9757
No log	0.7667	46	1.1169	0.0983	1.1169	1.0568
No log	0.8	48	1.1496	0.0987	1.1496	1.0722
No log	0.8333	50	1.0273	0.0982	1.0273	1.0135
No log	0.8667	52	0.9696	0.1252	0.9696	0.9847
No log	0.9	54	0.8319	0.2098	0.8319	0.9121
No log	0.9333	56	0.8092	0.2373	0.8092	0.8996
No log	0.9667	58	0.7802	0.2366	0.7802	0.8833
No log	1.0	60	0.8119	0.1803	0.8119	0.9011
No log	1.0333	62	0.9993	0.1819	0.9993	0.9996
No log	1.0667	64	0.9980	0.2097	0.9980	0.9990
No log	1.1	66	0.8793	0.1544	0.8793	0.9377
No log	1.1333	68	0.9269	0.2222	0.9269	0.9627
No log	1.1667	70	1.0744	0.2335	1.0744	1.0365
No log	1.2	72	1.0542	0.2354	1.0542	1.0267
No log	1.2333	74	0.8663	0.2156	0.8663	0.9307
No log	1.2667	76	0.7267	0.1094	0.7267	0.8524
No log	1.3	78	0.7203	0.0757	0.7203	0.8487
No log	1.3333	80	0.7365	0.0	0.7365	0.8582
No log	1.3667	82	0.7717	0.0940	0.7717	0.8785
No log	1.4	84	0.8323	0.1770	0.8323	0.9123
No log	1.4333	86	0.8829	0.1649	0.8829	0.9396
No log	1.4667	88	0.9361	0.1973	0.9361	0.9675
No log	1.5	90	0.9609	0.1957	0.9609	0.9803
No log	1.5333	92	1.0521	0.1565	1.0521	1.0257
No log	1.5667	94	1.1695	0.0726	1.1695	1.0814
No log	1.6	96	1.2176	-0.0399	1.2176	1.1035
No log	1.6333	98	1.1680	-0.0629	1.1680	1.0807
No log	1.6667	100	1.1805	0.0580	1.1805	1.0865
No log	1.7	102	1.1908	0.0514	1.1908	1.0912
No log	1.7333	104	1.2459	-0.0095	1.2459	1.1162
No log	1.7667	106	1.5369	0.0328	1.5369	1.2397
No log	1.8	108	1.4588	0.0328	1.4588	1.2078
No log	1.8333	110	1.1885	0.0759	1.1885	1.0902
No log	1.8667	112	0.9453	0.2379	0.9453	0.9723
No log	1.9	114	0.8923	0.0927	0.8923	0.9446
No log	1.9333	116	0.9336	0.1766	0.9336	0.9662
No log	1.9667	118	0.8782	0.1815	0.8782	0.9371
No log	2.0	120	0.7887	0.1359	0.7887	0.8881
No log	2.0333	122	0.7552	0.2181	0.7552	0.8690
No log	2.0667	124	0.7527	0.2024	0.7527	0.8676
No log	2.1	126	0.8084	0.2087	0.8084	0.8991
No log	2.1333	128	0.8221	0.3230	0.8221	0.9067
No log	2.1667	130	0.7954	0.2353	0.7954	0.8919
No log	2.2	132	0.7826	0.2607	0.7826	0.8847
No log	2.2333	134	0.8112	0.2193	0.8112	0.9007
No log	2.2667	136	0.8438	0.1373	0.8438	0.9186
No log	2.3	138	0.8648	0.2043	0.8648	0.9300
No log	2.3333	140	0.9049	0.2035	0.9049	0.9512
No log	2.3667	142	0.9038	0.2035	0.9038	0.9507
No log	2.4	144	0.8671	0.2273	0.8671	0.9312
No log	2.4333	146	0.8781	0.2442	0.8781	0.9370
No log	2.4667	148	0.9293	0.3105	0.9293	0.9640
No log	2.5	150	1.1173	0.2063	1.1173	1.0570
No log	2.5333	152	1.1975	0.1641	1.1975	1.0943
No log	2.5667	154	1.1136	0.1924	1.1136	1.0553
No log	2.6	156	0.9386	0.3074	0.9386	0.9688
No log	2.6333	158	0.8062	0.3496	0.8062	0.8979
No log	2.6667	160	0.7259	0.1580	0.7259	0.8520
No log	2.7	162	0.6974	0.1181	0.6974	0.8351
No log	2.7333	164	0.6877	0.1184	0.6877	0.8293
No log	2.7667	166	0.6835	0.1962	0.6835	0.8268
No log	2.8	168	0.6860	0.3061	0.6860	0.8282
No log	2.8333	170	0.7069	0.3061	0.7069	0.8408
No log	2.8667	172	0.7092	0.3302	0.7092	0.8421
No log	2.9	174	0.6950	0.2947	0.6950	0.8336
No log	2.9333	176	0.6958	0.2451	0.6958	0.8341
No log	2.9667	178	0.7080	0.2842	0.7080	0.8415
No log	3.0	180	0.7325	0.3754	0.7325	0.8559
No log	3.0333	182	0.7307	0.3390	0.7307	0.8548
No log	3.0667	184	0.7097	0.3031	0.7097	0.8424
No log	3.1	186	0.7071	0.2872	0.7071	0.8409
No log	3.1333	188	0.6888	0.2509	0.6888	0.8300
No log	3.1667	190	0.7057	0.3243	0.7057	0.8401
No log	3.2	192	0.7057	0.3366	0.7057	0.8400
No log	3.2333	194	0.6901	0.2243	0.6901	0.8307
No log	3.2667	196	0.6850	0.2243	0.6850	0.8276
No log	3.3	198	0.6845	0.2890	0.6845	0.8274
No log	3.3333	200	0.6741	0.2476	0.6741	0.8210
No log	3.3667	202	0.6659	0.4938	0.6659	0.8160
No log	3.4	204	0.6673	0.4788	0.6673	0.8169
No log	3.4333	206	0.6856	0.4351	0.6856	0.8280
No log	3.4667	208	0.6932	0.4548	0.6932	0.8326
No log	3.5	210	0.6870	0.5083	0.6870	0.8289
No log	3.5333	212	0.7159	0.4182	0.7159	0.8461
No log	3.5667	214	0.7583	0.3851	0.7583	0.8708
No log	3.6	216	0.7718	0.3617	0.7718	0.8785
No log	3.6333	218	0.7527	0.3851	0.7527	0.8676
No log	3.6667	220	0.7405	0.3899	0.7405	0.8606
No log	3.7	222	0.6390	0.4698	0.6390	0.7994
No log	3.7333	224	0.6156	0.3780	0.6156	0.7846
No log	3.7667	226	0.6112	0.4019	0.6112	0.7818
No log	3.8	228	0.6206	0.4044	0.6206	0.7878
No log	3.8333	230	0.6431	0.4345	0.6431	0.8019
No log	3.8667	232	0.6464	0.4262	0.6464	0.8040
No log	3.9	234	0.6410	0.4418	0.6410	0.8006
No log	3.9333	236	0.6436	0.4094	0.6436	0.8022
No log	3.9667	238	0.6665	0.3506	0.6665	0.8164
No log	4.0	240	0.6684	0.3643	0.6684	0.8176
No log	4.0333	242	0.6775	0.3701	0.6775	0.8231
No log	4.0667	244	0.6544	0.4360	0.6544	0.8090
No log	4.1	246	0.6316	0.4888	0.6316	0.7948
No log	4.1333	248	0.6231	0.4977	0.6231	0.7894
No log	4.1667	250	0.6097	0.5189	0.6097	0.7808
No log	4.2	252	0.6034	0.4224	0.6034	0.7768
No log	4.2333	254	0.6049	0.4224	0.6049	0.7778
No log	4.2667	256	0.6105	0.4224	0.6105	0.7814
No log	4.3	258	0.6230	0.4938	0.6230	0.7893
No log	4.3333	260	0.6178	0.4938	0.6178	0.7860
No log	4.3667	262	0.6042	0.5397	0.6042	0.7773
No log	4.4	264	0.6451	0.4227	0.6451	0.8032
No log	4.4333	266	0.6410	0.4924	0.6410	0.8006
No log	4.4667	268	0.6287	0.5389	0.6287	0.7929
No log	4.5	270	0.6341	0.4526	0.6341	0.7963
No log	4.5333	272	0.6207	0.4675	0.6207	0.7878
No log	4.5667	274	0.6054	0.5434	0.6054	0.7781
No log	4.6	276	0.6838	0.5053	0.6838	0.8269
No log	4.6333	278	0.8358	0.4479	0.8358	0.9142
No log	4.6667	280	0.8160	0.3929	0.8160	0.9034
No log	4.7	282	0.6731	0.4302	0.6731	0.8204
No log	4.7333	284	0.6295	0.5092	0.6295	0.7934
No log	4.7667	286	0.7032	0.4926	0.7032	0.8386
No log	4.8	288	0.7238	0.4703	0.7238	0.8508
No log	4.8333	290	0.6341	0.5048	0.6341	0.7963
No log	4.8667	292	0.5633	0.6196	0.5633	0.7506
No log	4.9	294	0.5473	0.5493	0.5473	0.7398
No log	4.9333	296	0.5501	0.4746	0.5501	0.7417
No log	4.9667	298	0.5738	0.5301	0.5738	0.7575
No log	5.0	300	0.6502	0.4395	0.6502	0.8064
No log	5.0333	302	0.6866	0.4395	0.6866	0.8286
No log	5.0667	304	0.6945	0.5112	0.6945	0.8334
No log	5.1	306	0.6326	0.4949	0.6326	0.7954
No log	5.1333	308	0.6147	0.5592	0.6147	0.7840
No log	5.1667	310	0.6077	0.5286	0.6077	0.7796
No log	5.2	312	0.6160	0.4874	0.6160	0.7848
No log	5.2333	314	0.6241	0.4591	0.6241	0.7900
No log	5.2667	316	0.6221	0.5142	0.6221	0.7887
No log	5.3	318	0.6145	0.5084	0.6145	0.7839
No log	5.3333	320	0.6083	0.4813	0.6083	0.7800
No log	5.3667	322	0.6041	0.4402	0.6041	0.7773
No log	5.4	324	0.6113	0.4802	0.6113	0.7819
No log	5.4333	326	0.6027	0.4858	0.6027	0.7764
No log	5.4667	328	0.6145	0.4795	0.6145	0.7839
No log	5.5	330	0.6156	0.4620	0.6156	0.7846
No log	5.5333	332	0.6065	0.4655	0.6065	0.7788
No log	5.5667	334	0.6117	0.5283	0.6117	0.7821
No log	5.6	336	0.6047	0.5046	0.6047	0.7776
No log	5.6333	338	0.5771	0.5432	0.5771	0.7597
No log	5.6667	340	0.5633	0.4782	0.5633	0.7506
No log	5.7	342	0.5514	0.4809	0.5514	0.7426
No log	5.7333	344	0.5459	0.5195	0.5459	0.7389
No log	5.7667	346	0.5538	0.5003	0.5538	0.7442
No log	5.8	348	0.5736	0.6092	0.5736	0.7574
No log	5.8333	350	0.5798	0.5583	0.5798	0.7614
No log	5.8667	352	0.5474	0.5543	0.5474	0.7398
No log	5.9	354	0.5173	0.5357	0.5173	0.7192
No log	5.9333	356	0.5000	0.6156	0.5000	0.7071
No log	5.9667	358	0.4990	0.6269	0.4990	0.7064
No log	6.0	360	0.5267	0.5014	0.5267	0.7257
No log	6.0333	362	0.5232	0.5014	0.5232	0.7233
No log	6.0667	364	0.5080	0.5266	0.5080	0.7128
No log	6.1	366	0.5055	0.5860	0.5055	0.7110
No log	6.1333	368	0.5158	0.5373	0.5158	0.7182
No log	6.1667	370	0.5240	0.5160	0.5240	0.7239
No log	6.2	372	0.5422	0.5390	0.5422	0.7363
No log	6.2333	374	0.5494	0.5234	0.5494	0.7412
No log	6.2667	376	0.5373	0.6092	0.5373	0.7330
No log	6.3	378	0.5482	0.5322	0.5482	0.7404
No log	6.3333	380	0.5696	0.5322	0.5696	0.7547
No log	6.3667	382	0.5671	0.5528	0.5671	0.7531
No log	6.4	384	0.5651	0.5923	0.5651	0.7517
No log	6.4333	386	0.5742	0.6529	0.5742	0.7577
No log	6.4667	388	0.5931	0.6434	0.5931	0.7702
No log	6.5	390	0.6497	0.5787	0.6497	0.8060
No log	6.5333	392	0.7255	0.4476	0.7255	0.8518
No log	6.5667	394	0.6268	0.5442	0.6268	0.7917
No log	6.6	396	0.5615	0.5150	0.5615	0.7494
No log	6.6333	398	0.5473	0.5373	0.5473	0.7398
No log	6.6667	400	0.5434	0.5248	0.5434	0.7371
No log	6.7	402	0.5515	0.5110	0.5515	0.7426
No log	6.7333	404	0.5332	0.5110	0.5332	0.7302
No log	6.7667	406	0.5268	0.6265	0.5268	0.7258
No log	6.8	408	0.6154	0.4684	0.6154	0.7844
No log	6.8333	410	0.7014	0.4230	0.7014	0.8375
No log	6.8667	412	0.6903	0.4230	0.6903	0.8309
No log	6.9	414	0.6367	0.4470	0.6367	0.7980
No log	6.9333	416	0.5833	0.4437	0.5833	0.7637
No log	6.9667	418	0.5601	0.3728	0.5601	0.7484
No log	7.0	420	0.5833	0.4698	0.5833	0.7638
No log	7.0333	422	0.5696	0.4507	0.5696	0.7547
No log	7.0667	424	0.5405	0.5430	0.5405	0.7352
No log	7.1	426	0.5524	0.4617	0.5524	0.7432
No log	7.1333	428	0.5508	0.5167	0.5508	0.7422
No log	7.1667	430	0.5533	0.4611	0.5533	0.7438
No log	7.2	432	0.5417	0.4829	0.5417	0.7360
No log	7.2333	434	0.5388	0.4767	0.5388	0.7340
No log	7.2667	436	0.5543	0.4841	0.5543	0.7445
No log	7.3	438	0.5500	0.4841	0.5500	0.7417
No log	7.3333	440	0.5279	0.4517	0.5279	0.7266
No log	7.3667	442	0.5121	0.4949	0.5121	0.7156
No log	7.4	444	0.5132	0.4425	0.5132	0.7164
No log	7.4333	446	0.5104	0.5122	0.5104	0.7144
No log	7.4667	448	0.5168	0.4273	0.5168	0.7189
No log	7.5	450	0.5311	0.4	0.5311	0.7288
No log	7.5333	452	0.5941	0.4351	0.5941	0.7708
No log	7.5667	454	0.6135	0.4630	0.6135	0.7833
No log	7.6	456	0.5811	0.4610	0.5811	0.7623
No log	7.6333	458	0.5591	0.4589	0.5591	0.7477
No log	7.6667	460	0.5206	0.4938	0.5206	0.7215
No log	7.7	462	0.5128	0.4817	0.5128	0.7161
No log	7.7333	464	0.5172	0.5784	0.5172	0.7192
No log	7.7667	466	0.5652	0.4948	0.5652	0.7518
No log	7.8	468	0.5717	0.4315	0.5717	0.7561
No log	7.8333	470	0.5417	0.5133	0.5417	0.7360
No log	7.8667	472	0.5374	0.5133	0.5374	0.7331
No log	7.9	474	0.5364	0.5488	0.5364	0.7324
No log	7.9333	476	0.5470	0.5593	0.5470	0.7396
No log	7.9667	478	0.5386	0.5428	0.5386	0.7339
No log	8.0	480	0.5177	0.6024	0.5177	0.7195
No log	8.0333	482	0.5161	0.6229	0.5161	0.7184
No log	8.0667	484	0.5189	0.5765	0.5189	0.7203
No log	8.1	486	0.5328	0.5574	0.5328	0.7299
No log	8.1333	488	0.5417	0.5574	0.5417	0.7360
No log	8.1667	490	0.5327	0.4970	0.5327	0.7299
No log	8.2	492	0.5329	0.5648	0.5329	0.7300
No log	8.2333	494	0.5433	0.5501	0.5433	0.7371
No log	8.2667	496	0.5239	0.6039	0.5239	0.7238
No log	8.3	498	0.5235	0.5750	0.5235	0.7235
0.3938	8.3333	500	0.5221	0.5860	0.5221	0.7226
0.3938	8.3667	502	0.5211	0.6129	0.5211	0.7219
0.3938	8.4	504	0.5234	0.6530	0.5234	0.7235
0.3938	8.4333	506	0.5443	0.4724	0.5443	0.7378
0.3938	8.4667	508	0.5848	0.4165	0.5848	0.7647
0.3938	8.5	510	0.5547	0.4186	0.5547	0.7448
0.3938	8.5333	512	0.5054	0.6096	0.5054	0.7109
0.3938	8.5667	514	0.5001	0.5414	0.5001	0.7072
0.3938	8.6	516	0.5011	0.5414	0.5011	0.7079
0.3938	8.6333	518	0.5110	0.5649	0.5110	0.7149
0.3938	8.6667	520	0.5168	0.5414	0.5168	0.7189
0.3938	8.7	522	0.4984	0.5079	0.4984	0.7060

Framework versions

—Transformers 4.44.2
—Pytorch 2.4.0+cu118
—Datasets 2.21.0
—Tokenizers 0.19.1