Modelpublic

MayBashendy/Arabic_CrossPrompt_FineTuningAraBERT_noAug_TestTask2_grammar

sourceHugging Faceupdated 2y agoView on Hugging Face

0likes2downloads

Model Card

ArabicCrossPromptFineTuningAraBERTnoAugTestTask2_grammar

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

—Loss: 0.6030
—Qwk: 0.4655
—Mse: 0.6030
—Rmse: 0.7765

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 2e-05
—trainbatchsize: 8
—evalbatchsize: 8
—seed: 42
—optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
—lrschedulertype: linear
—num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse	Rmse
No log	0.0213	2	3.8353	0.0201	3.8353	1.9584
No log	0.0426	4	2.9300	0.0458	2.9300	1.7117
No log	0.0638	6	1.2632	0.0935	1.2632	1.1239
No log	0.0851	8	0.7292	0.2269	0.7292	0.8539
No log	0.1064	10	0.8792	0.0645	0.8792	0.9376
No log	0.1277	12	0.7469	0.1712	0.7469	0.8643
No log	0.1489	14	0.6919	0.2187	0.6919	0.8318
No log	0.1702	16	0.6642	0.3058	0.6642	0.8150
No log	0.1915	18	0.6236	0.3770	0.6236	0.7897
No log	0.2128	20	0.5946	0.4361	0.5946	0.7711
No log	0.2340	22	0.6425	0.3128	0.6425	0.8015
No log	0.2553	24	0.6332	0.2684	0.6332	0.7957
No log	0.2766	26	0.6569	0.3182	0.6569	0.8105
No log	0.2979	28	0.5891	0.4035	0.5891	0.7676
No log	0.3191	30	0.5673	0.4249	0.5673	0.7532
No log	0.3404	32	0.5691	0.4040	0.5691	0.7544
No log	0.3617	34	0.5366	0.4332	0.5366	0.7325
No log	0.3830	36	0.5657	0.4203	0.5657	0.7521
No log	0.4043	38	0.6357	0.3282	0.6357	0.7973
No log	0.4255	40	0.6498	0.3394	0.6498	0.8061
No log	0.4468	42	0.8192	0.0680	0.8192	0.9051
No log	0.4681	44	0.9016	-0.0504	0.9016	0.9495
No log	0.4894	46	0.8537	0.0138	0.8537	0.9240
No log	0.5106	48	0.6766	0.3085	0.6766	0.8225
No log	0.5319	50	0.5221	0.4931	0.5221	0.7225
No log	0.5532	52	0.5309	0.5142	0.5309	0.7286
No log	0.5745	54	0.5185	0.5175	0.5185	0.7201
No log	0.5957	56	0.5916	0.4713	0.5916	0.7692
No log	0.6170	58	0.6543	0.3639	0.6543	0.8089
No log	0.6383	60	0.6571	0.3485	0.6571	0.8106
No log	0.6596	62	0.5339	0.3765	0.5339	0.7307
No log	0.6809	64	0.4673	0.5524	0.4673	0.6836
No log	0.7021	66	0.4759	0.5650	0.4759	0.6899
No log	0.7234	68	0.5051	0.5182	0.5051	0.7107
No log	0.7447	70	0.6184	0.4755	0.6184	0.7864
No log	0.7660	72	0.6518	0.4735	0.6518	0.8073
No log	0.7872	74	0.5901	0.5537	0.5901	0.7682
No log	0.8085	76	0.5770	0.5099	0.5770	0.7596
No log	0.8298	78	0.6664	0.4271	0.6664	0.8164
No log	0.8511	80	0.7024	0.4139	0.7024	0.8381
No log	0.8723	82	0.6255	0.4808	0.6255	0.7909
No log	0.8936	84	0.5437	0.5660	0.5437	0.7374
No log	0.9149	86	0.4890	0.6093	0.4890	0.6993
No log	0.9362	88	0.4739	0.6140	0.4739	0.6884
No log	0.9574	90	0.4791	0.6009	0.4791	0.6922
No log	0.9787	92	0.4738	0.5986	0.4738	0.6883
No log	1.0	94	0.4748	0.5682	0.4748	0.6891
No log	1.0213	96	0.5206	0.4763	0.5206	0.7215
No log	1.0426	98	0.5713	0.4223	0.5713	0.7558
No log	1.0638	100	0.5945	0.3720	0.5945	0.7710
No log	1.0851	102	0.5834	0.4060	0.5834	0.7638
No log	1.1064	104	0.5643	0.4751	0.5643	0.7512
No log	1.1277	106	0.6183	0.4209	0.6183	0.7863
No log	1.1489	108	0.6264	0.4433	0.6264	0.7915
No log	1.1702	110	0.5943	0.4850	0.5943	0.7709
No log	1.1915	112	0.5699	0.4781	0.5699	0.7549
No log	1.2128	114	0.5515	0.4878	0.5515	0.7426
No log	1.2340	116	0.5552	0.4979	0.5552	0.7451
No log	1.2553	118	0.5574	0.4925	0.5574	0.7466
No log	1.2766	120	0.5590	0.4562	0.5590	0.7477
No log	1.2979	122	0.5637	0.4266	0.5637	0.7508
No log	1.3191	124	0.5795	0.4197	0.5795	0.7612
No log	1.3404	126	0.5856	0.4182	0.5856	0.7652
No log	1.3617	128	0.5481	0.4212	0.5481	0.7404
No log	1.3830	130	0.5419	0.4693	0.5419	0.7361
No log	1.4043	132	0.5646	0.4130	0.5646	0.7514
No log	1.4255	134	0.6854	0.3461	0.6854	0.8279
No log	1.4468	136	0.7330	0.2659	0.7330	0.8562
No log	1.4681	138	0.7282	0.2204	0.7282	0.8534
No log	1.4894	140	0.5949	0.3650	0.5949	0.7713
No log	1.5106	142	0.5636	0.4481	0.5636	0.7507
No log	1.5319	144	0.5544	0.4549	0.5544	0.7446
No log	1.5532	146	0.5737	0.3918	0.5737	0.7575
No log	1.5745	148	0.8020	0.1292	0.8020	0.8956
No log	1.5957	150	0.9233	0.0850	0.9233	0.9609
No log	1.6170	152	0.7684	0.3153	0.7684	0.8766
No log	1.6383	154	0.5874	0.5328	0.5874	0.7664
No log	1.6596	156	0.5509	0.5484	0.5509	0.7422
No log	1.6809	158	0.7366	0.3269	0.7366	0.8582
No log	1.7021	160	0.7420	0.2354	0.7420	0.8614
No log	1.7234	162	0.5078	0.4948	0.5078	0.7126
No log	1.7447	164	0.4810	0.5495	0.4810	0.6935
No log	1.7660	166	0.4906	0.5477	0.4906	0.7004
No log	1.7872	168	0.4594	0.5177	0.4594	0.6778
No log	1.8085	170	0.5333	0.4764	0.5333	0.7303
No log	1.8298	172	0.6349	0.4132	0.6349	0.7968
No log	1.8511	174	0.7694	0.4500	0.7694	0.8772
No log	1.8723	176	0.8621	0.4338	0.8621	0.9285
No log	1.8936	178	0.8167	0.4688	0.8167	0.9037
No log	1.9149	180	0.7867	0.4777	0.7867	0.8869
No log	1.9362	182	0.7318	0.5072	0.7318	0.8555
No log	1.9574	184	0.8027	0.4968	0.8027	0.8959
No log	1.9787	186	0.6452	0.5350	0.6452	0.8033
No log	2.0	188	0.5025	0.5527	0.5025	0.7088
No log	2.0213	190	0.5075	0.5484	0.5075	0.7124
No log	2.0426	192	0.5151	0.5493	0.5151	0.7177
No log	2.0638	194	0.5640	0.5602	0.5640	0.7510
No log	2.0851	196	0.4783	0.6011	0.4783	0.6916
No log	2.1064	198	0.4742	0.5927	0.4742	0.6886
No log	2.1277	200	0.4825	0.5690	0.4825	0.6946
No log	2.1489	202	0.5120	0.4867	0.5120	0.7156
No log	2.1702	204	0.5162	0.4621	0.5162	0.7185
No log	2.1915	206	0.4929	0.4656	0.4929	0.7021
No log	2.2128	208	0.5061	0.4925	0.5061	0.7114
No log	2.2340	210	0.6116	0.5030	0.6116	0.7821
No log	2.2553	212	0.6582	0.4890	0.6582	0.8113
No log	2.2766	214	0.6269	0.5382	0.6269	0.7918
No log	2.2979	216	0.5930	0.5496	0.5930	0.7701
No log	2.3191	218	0.5960	0.5525	0.5960	0.7720
No log	2.3404	220	0.6056	0.5805	0.6056	0.7782
No log	2.3617	222	0.6127	0.5794	0.6127	0.7827
No log	2.3830	224	0.6947	0.4956	0.6947	0.8335
No log	2.4043	226	0.7535	0.4206	0.7535	0.8680
No log	2.4255	228	0.7554	0.3611	0.7554	0.8691
No log	2.4468	230	0.7373	0.3611	0.7373	0.8586
No log	2.4681	232	0.6295	0.4773	0.6295	0.7934
No log	2.4894	234	0.4964	0.5674	0.4964	0.7045
No log	2.5106	236	0.5946	0.5271	0.5946	0.7711
No log	2.5319	238	0.6252	0.5035	0.6252	0.7907
No log	2.5532	240	0.5040	0.5970	0.5040	0.7100
No log	2.5745	242	0.4640	0.5599	0.4640	0.6812
No log	2.5957	244	0.5576	0.4241	0.5576	0.7467
No log	2.6170	246	0.6953	0.3694	0.6953	0.8338
No log	2.6383	248	0.7648	0.3798	0.7648	0.8745
No log	2.6596	250	0.7191	0.4558	0.7191	0.8480
No log	2.6809	252	0.5806	0.5656	0.5806	0.7619
No log	2.7021	254	0.4949	0.5919	0.4949	0.7035
No log	2.7234	256	0.5325	0.6003	0.5325	0.7297
No log	2.7447	258	0.5309	0.5945	0.5309	0.7286
No log	2.7660	260	0.4762	0.6384	0.4762	0.6901
No log	2.7872	262	0.4551	0.6425	0.4551	0.6746
No log	2.8085	264	0.5098	0.5572	0.5098	0.7140
No log	2.8298	266	0.6526	0.4546	0.6526	0.8078
No log	2.8511	268	0.6799	0.4008	0.6799	0.8246
No log	2.8723	270	0.5843	0.4535	0.5843	0.7644
No log	2.8936	272	0.4450	0.5958	0.4450	0.6671
No log	2.9149	274	0.4175	0.6161	0.4175	0.6461
No log	2.9362	276	0.4276	0.6040	0.4276	0.6539
No log	2.9574	278	0.4181	0.5851	0.4181	0.6466
No log	2.9787	280	0.4592	0.4783	0.4592	0.6776
No log	3.0	282	0.6471	0.2913	0.6471	0.8044
No log	3.0213	284	0.7901	0.2220	0.7901	0.8889
No log	3.0426	286	0.7880	0.1960	0.7880	0.8877
No log	3.0638	288	0.7042	0.2531	0.7042	0.8391
No log	3.0851	290	0.5986	0.3632	0.5986	0.7737
No log	3.1064	292	0.4788	0.5344	0.4788	0.6920
No log	3.1277	294	0.4686	0.6116	0.4686	0.6846
No log	3.1489	296	0.5117	0.6111	0.5117	0.7153
No log	3.1702	298	0.5134	0.6114	0.5134	0.7165
No log	3.1915	300	0.6089	0.5676	0.6089	0.7803
No log	3.2128	302	0.7372	0.4511	0.7372	0.8586
No log	3.2340	304	0.8497	0.3395	0.8497	0.9218
No log	3.2553	306	0.7684	0.3668	0.7684	0.8766
No log	3.2766	308	0.6282	0.3868	0.6282	0.7926
No log	3.2979	310	0.5407	0.3947	0.5407	0.7353
No log	3.3191	312	0.5210	0.4589	0.5210	0.7218
No log	3.3404	314	0.4896	0.5439	0.4896	0.6997
No log	3.3617	316	0.4538	0.6260	0.4538	0.6736
No log	3.3830	318	0.4815	0.6149	0.4815	0.6939
No log	3.4043	320	0.5970	0.4958	0.5970	0.7727
No log	3.4255	322	0.6804	0.5261	0.6804	0.8248
No log	3.4468	324	0.7530	0.5092	0.7530	0.8677
No log	3.4681	326	0.7599	0.5106	0.7599	0.8717
No log	3.4894	328	0.6928	0.5204	0.6928	0.8323
No log	3.5106	330	0.5752	0.5461	0.5752	0.7584
No log	3.5319	332	0.5015	0.5647	0.5015	0.7082
No log	3.5532	334	0.4821	0.5433	0.4821	0.6944
No log	3.5745	336	0.5275	0.5172	0.5275	0.7263
No log	3.5957	338	0.5469	0.5323	0.5469	0.7396
No log	3.6170	340	0.5212	0.5695	0.5212	0.7220
No log	3.6383	342	0.4912	0.6060	0.4912	0.7009
No log	3.6596	344	0.5048	0.6126	0.5048	0.7105
No log	3.6809	346	0.5924	0.5724	0.5924	0.7697
No log	3.7021	348	0.7037	0.4895	0.7037	0.8389
No log	3.7234	350	0.6837	0.4321	0.6837	0.8269
No log	3.7447	352	0.5887	0.4928	0.5887	0.7673
No log	3.7660	354	0.5224	0.5072	0.5224	0.7228
No log	3.7872	356	0.4770	0.5110	0.4770	0.6907
No log	3.8085	358	0.4662	0.5444	0.4662	0.6828
No log	3.8298	360	0.4790	0.5386	0.4790	0.6921
No log	3.8511	362	0.4895	0.5736	0.4895	0.6996
No log	3.8723	364	0.5308	0.5799	0.5308	0.7286
No log	3.8936	366	0.5652	0.6187	0.5652	0.7518
No log	3.9149	368	0.5623	0.6159	0.5623	0.7499
No log	3.9362	370	0.5082	0.6095	0.5082	0.7129
No log	3.9574	372	0.5170	0.5907	0.5170	0.7190
No log	3.9787	374	0.7160	0.4793	0.7160	0.8462
No log	4.0	376	0.8899	0.3589	0.8899	0.9434
No log	4.0213	378	0.8667	0.3664	0.8667	0.9310
No log	4.0426	380	0.6435	0.4517	0.6435	0.8022
No log	4.0638	382	0.4468	0.6170	0.4468	0.6684
No log	4.0851	384	0.4513	0.5416	0.4513	0.6718
No log	4.1064	386	0.4389	0.5934	0.4389	0.6625
No log	4.1277	388	0.4779	0.5591	0.4779	0.6913
No log	4.1489	390	0.5858	0.5187	0.5858	0.7654
No log	4.1702	392	0.7549	0.4731	0.7549	0.8689
No log	4.1915	394	0.7766	0.4648	0.7766	0.8813
No log	4.2128	396	0.7079	0.4875	0.7079	0.8413
No log	4.2340	398	0.5701	0.5573	0.5701	0.7551
No log	4.2553	400	0.4573	0.6223	0.4573	0.6763
No log	4.2766	402	0.4413	0.6215	0.4413	0.6643
No log	4.2979	404	0.4637	0.5474	0.4637	0.6810
No log	4.3191	406	0.5566	0.4797	0.5566	0.7460
No log	4.3404	408	0.6238	0.3833	0.6238	0.7898
No log	4.3617	410	0.6535	0.3787	0.6535	0.8084
No log	4.3830	412	0.6120	0.4413	0.6120	0.7823
No log	4.4043	414	0.5500	0.5264	0.5500	0.7416
No log	4.4255	416	0.5426	0.5563	0.5426	0.7366
No log	4.4468	418	0.5207	0.5783	0.5207	0.7216
No log	4.4681	420	0.5292	0.5463	0.5292	0.7275
No log	4.4894	422	0.5117	0.5554	0.5117	0.7153
No log	4.5106	424	0.5026	0.5435	0.5026	0.7089
No log	4.5319	426	0.4773	0.5688	0.4773	0.6908
No log	4.5532	428	0.4916	0.5730	0.4916	0.7012
No log	4.5745	430	0.4653	0.5841	0.4653	0.6821
No log	4.5957	432	0.4730	0.5992	0.4730	0.6878
No log	4.6170	434	0.5361	0.5815	0.5361	0.7322
No log	4.6383	436	0.7194	0.4948	0.7194	0.8482
No log	4.6596	438	0.8576	0.4359	0.8576	0.9261
No log	4.6809	440	0.7990	0.4571	0.7990	0.8938
No log	4.7021	442	0.6099	0.5264	0.6099	0.7810
No log	4.7234	444	0.4672	0.6245	0.4672	0.6835
No log	4.7447	446	0.4764	0.6350	0.4764	0.6902
No log	4.7660	448	0.4937	0.6232	0.4937	0.7026
No log	4.7872	450	0.5072	0.6095	0.5072	0.7122
No log	4.8085	452	0.5516	0.5814	0.5516	0.7427
No log	4.8298	454	0.6285	0.4976	0.6285	0.7928
No log	4.8511	456	0.6398	0.4528	0.6398	0.7999
No log	4.8723	458	0.6480	0.3935	0.6480	0.8050
No log	4.8936	460	0.5995	0.4155	0.5995	0.7743
No log	4.9149	462	0.5271	0.4843	0.5271	0.7260
No log	4.9362	464	0.5067	0.5427	0.5067	0.7118
No log	4.9574	466	0.5518	0.5843	0.5518	0.7428
No log	4.9787	468	0.6820	0.5309	0.6820	0.8258
No log	5.0	470	0.8470	0.5087	0.8470	0.9203
No log	5.0213	472	0.8969	0.5077	0.8969	0.9471
No log	5.0426	474	0.8190	0.4974	0.8190	0.9050
No log	5.0638	476	0.6492	0.5148	0.6492	0.8057
No log	5.0851	478	0.5597	0.5491	0.5597	0.7481
No log	5.1064	480	0.5568	0.5039	0.5568	0.7462
No log	5.1277	482	0.5314	0.4720	0.5314	0.7290
No log	5.1489	484	0.5333	0.4822	0.5333	0.7303
No log	5.1702	486	0.5786	0.5050	0.5786	0.7607
No log	5.1915	488	0.5877	0.5592	0.5877	0.7666
No log	5.2128	490	0.5367	0.5924	0.5367	0.7326
No log	5.2340	492	0.5015	0.5998	0.5015	0.7082
No log	5.2553	494	0.4655	0.5871	0.4655	0.6823
No log	5.2766	496	0.4718	0.5953	0.4718	0.6869
No log	5.2979	498	0.4624	0.6063	0.4624	0.6800
0.4944	5.3191	500	0.5517	0.5755	0.5517	0.7428
0.4944	5.3404	502	0.7229	0.4428	0.7229	0.8502
0.4944	5.3617	504	0.8184	0.4186	0.8184	0.9046
0.4944	5.3830	506	0.7250	0.4853	0.7250	0.8515
0.4944	5.4043	508	0.5547	0.5367	0.5547	0.7448
0.4944	5.4255	510	0.4673	0.5883	0.4673	0.6836
0.4944	5.4468	512	0.4649	0.5996	0.4649	0.6818
0.4944	5.4681	514	0.4657	0.5954	0.4657	0.6824
0.4944	5.4894	516	0.5302	0.5747	0.5302	0.7281
0.4944	5.5106	518	0.7249	0.4764	0.7249	0.8514
0.4944	5.5319	520	0.8394	0.4032	0.8394	0.9162
0.4944	5.5532	522	0.7500	0.4129	0.7500	0.8660
0.4944	5.5745	524	0.6030	0.4655	0.6030	0.7765

Framework versions

—Transformers 4.44.2
—Pytorch 2.4.0+cu118
—Datasets 2.21.0
—Tokenizers 0.19.1