Modelpublic

cportoca/MMS_Quechua_finetuned

sourceHugging Facecc-by-nc-4.0updated 2y agoView on Hugging Face

0likes5downloads

Model Card

MMSQuechuafinetuned

This model is a fine-tuned version of facebook/mms-1b-all on an unknown dataset. It achieves the following results on the evaluation set:

—Loss: 0.2531
—Wer: 0.3172

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 0.001
—trainbatchsize: 16
—evalbatchsize: 8
—seed: 42
—optimizer: Use adamwtorch with betas=(0.9,0.999) and epsilon=1e-08 and optimizerargs=No additional optimizer arguments
—lrschedulertype: linear
—lrschedulerwarmup_steps: 100
—num_epochs: 20
—mixedprecisiontraining: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
10.4014	0.1355	100	4.9727	0.9997
2.7516	0.2710	200	0.6059	0.5024
0.6736	0.4065	300	0.5783	0.4747
0.5917	0.5420	400	0.5185	0.4412
0.5776	0.6775	500	0.4926	0.4249
1.0538	0.8130	600	0.5035	0.4255
0.5216	0.9485	700	0.4767	0.4167
0.6692	1.0840	800	0.4524	0.4274
0.5159	1.2195	900	0.4474	0.4035
0.631	1.3550	1000	0.4456	0.4117
0.6377	1.4905	1100	0.4422	0.4104
0.7765	1.6260	1200	0.4571	0.4082
0.4849	1.7615	1300	0.4563	0.4026
0.4695	1.8970	1400	0.4385	0.4004
0.6693	2.0325	1500	0.4209	0.3928
0.6357	2.1680	1600	0.4203	0.3966
0.4671	2.3035	1700	0.4201	0.3994
0.4672	2.4390	1800	0.4208	0.4038
0.7265	2.5745	1900	0.4195	0.4098
0.4802	2.7100	2000	0.3781	0.3828
0.6319	2.8455	2100	0.3727	0.3844
0.5765	2.9810	2200	0.3976	0.3853
0.5579	3.1165	2300	0.3601	0.3850
0.4431	3.2520	2400	0.3513	0.3881
0.6378	3.3875	2500	0.3406	0.3693
0.6201	3.5230	2600	0.3366	0.3725
0.5285	3.6585	2700	0.3390	0.3731
0.573	3.7940	2800	0.3563	0.3750
0.3998	3.9295	2900	0.4177	0.3731
0.6257	4.0650	3000	0.3899	0.3800
0.5346	4.2005	3100	0.3567	0.3803
0.5731	4.3360	3200	0.3671	0.3866
0.5217	4.4715	3300	0.3429	0.3768
0.4091	4.6070	3400	0.3363	0.3822
0.6284	4.7425	3500	0.3727	0.3794
0.3642	4.8780	3600	0.3273	0.3834
0.4199	5.0136	3700	0.3299	0.3765
0.361	5.1491	3800	0.3164	0.3539
0.5072	5.2846	3900	0.3255	0.3640
0.584	5.4201	4000	0.3168	0.3681
0.7192	5.5556	4100	0.3266	0.3586
0.4023	5.6911	4200	0.3279	0.3765
0.3849	5.8266	4300	0.3274	0.3533
0.5499	5.9621	4400	0.3182	0.3546
0.505	6.0976	4500	0.3199	0.3590
0.3689	6.2331	4600	0.3168	0.3411
0.4963	6.3686	4700	0.3228	0.3455
0.4904	6.5041	4800	0.3248	0.3634
0.3871	6.6396	4900	0.3128	0.3555
0.5636	6.7751	5000	0.3129	0.3552
0.525	6.9106	5100	0.3089	0.3608
0.5762	7.0461	5200	0.3170	0.3527
0.3613	7.1816	5300	0.3156	0.3602
0.4433	7.3171	5400	0.3015	0.3612
0.3692	7.4526	5500	0.3228	0.3608
0.6615	7.5881	5600	0.3052	0.3561
0.4931	7.7236	5700	0.3039	0.3458
0.3608	7.8591	5800	0.3075	0.3464
0.4666	7.9946	5900	0.3047	0.3583
0.3236	8.1301	6000	0.3117	0.3574
0.6959	8.2656	6100	0.3431	0.3499
0.3459	8.4011	6200	0.3075	0.3517
0.4103	8.5366	6300	0.2924	0.3408
0.424	8.6721	6400	0.3148	0.3511
0.3373	8.8076	6500	0.3104	0.3473
0.4517	8.9431	6600	0.3218	0.3546
0.4533	9.0786	6700	0.3196	0.3514
0.4015	9.2141	6800	0.3088	0.3583
0.336	9.3496	6900	0.2927	0.3370
0.5446	9.4851	7000	0.2840	0.3430
0.4258	9.6206	7100	0.3002	0.3430
0.3432	9.7561	7200	0.2911	0.3486
0.3131	9.8916	7300	0.2907	0.3323
0.5729	10.0271	7400	0.2942	0.3326
0.3266	10.1626	7500	0.2914	0.3401
0.3512	10.2981	7600	0.2956	0.3414
0.6843	10.4336	7700	0.2840	0.3392
0.3667	10.5691	7800	0.2857	0.3348
0.3088	10.7046	7900	0.2888	0.3351
0.3679	10.8401	8000	0.2896	0.3361
0.319	10.9756	8100	0.2768	0.3320
0.3045	11.1111	8200	0.2810	0.3348
0.3169	11.2466	8300	0.2813	0.3307
0.3837	11.3821	8400	0.2831	0.3251
0.3687	11.5176	8500	0.2864	0.3351
0.322	11.6531	8600	0.2831	0.3216
0.565	11.7886	8700	0.2776	0.3348
0.363	11.9241	8800	0.2738	0.3270
0.3281	12.0596	8900	0.2785	0.3244
0.3626	12.1951	9000	0.2773	0.3414
0.3201	12.3306	9100	0.2748	0.3222
0.2993	12.4661	9200	0.2833	0.3251
0.5219	12.6016	9300	0.2936	0.3323
0.3078	12.7371	9400	0.2801	0.3329
0.3282	12.8726	9500	0.2890	0.3298
0.3013	13.0081	9600	0.2807	0.3285
0.2689	13.1436	9700	0.3006	0.3389
0.3119	13.2791	9800	0.2885	0.3310
0.3178	13.4146	9900	0.2816	0.3279
0.5885	13.5501	10000	0.2699	0.3188
0.3134	13.6856	10100	0.2857	0.3213
0.3355	13.8211	10200	0.2729	0.3175
0.296	13.9566	10300	0.2732	0.3229
0.3573	14.0921	10400	0.2699	0.3345
0.478	14.2276	10500	0.2692	0.3188
0.3013	14.3631	10600	0.2636	0.3179
0.2978	14.4986	10700	0.2641	0.3175
0.2753	14.6341	10800	0.2697	0.3169
0.3017	14.7696	10900	0.2688	0.3179
0.2897	14.9051	11000	0.2662	0.3135
0.2861	15.0407	11100	0.2650	0.3201
0.2752	15.1762	11200	0.2582	0.3153
0.2908	15.3117	11300	0.2645	0.3219
0.286	15.4472	11400	0.2647	0.3147
0.2828	15.5827	11500	0.2633	0.3169
0.4632	15.7182	11600	0.2628	0.3207
0.2994	15.8537	11700	0.2595	0.3160
0.3075	15.9892	11800	0.2616	0.3201
0.267	16.1247	11900	0.2628	0.3207
0.2825	16.2602	12000	0.2593	0.3191
0.2684	16.3957	12100	0.2554	0.3175
0.4811	16.5312	12200	0.2554	0.3298
0.2904	16.6667	12300	0.2574	0.3160
0.2781	16.8022	12400	0.2612	0.3166
0.2667	16.9377	12500	0.2597	0.3191
0.2945	17.0732	12600	0.2584	0.3150
0.2697	17.2087	12700	0.2546	0.3125
0.2726	17.3442	12800	0.2548	0.3141
0.2679	17.4797	12900	0.2586	0.3119
0.2762	17.6152	13000	0.2588	0.3131
0.2713	17.7507	13100	0.2563	0.3125
0.4666	17.8862	13200	0.2540	0.3125
0.2568	18.0217	13300	0.2613	0.3131
0.4632	18.1572	13400	0.2566	0.3182
0.2926	18.2927	13500	0.2553	0.3166
0.2743	18.4282	13600	0.2535	0.3166
0.2677	18.5637	13700	0.2566	0.3128
0.2763	18.6992	13800	0.2537	0.3125
0.2581	18.8347	13900	0.2550	0.3144
0.2476	18.9702	14000	0.2543	0.3131
0.254	19.1057	14100	0.2548	0.3131
0.2591	19.2412	14200	0.2558	0.3144
0.2728	19.3767	14300	0.2534	0.3147
0.2856	19.5122	14400	0.2523	0.3144
0.2596	19.6477	14500	0.2510	0.3119
0.4273	19.7832	14600	0.2521	0.3153
0.2559	19.9187	14700	0.2531	0.3172

Framework versions

—Transformers 4.46.3
—Pytorch 2.4.1+cu121
—Datasets 3.1.0
—Tokenizers 0.20.3