Modelpublic

OyellaA/W2V2_Bert_cdli_ugandan_english_nonstandard_speech_finetune_v1

sourceHugging Facemitupdated 6mo agoView on Hugging Face

0likes3downloads

Model Card

W2V2Bertcdliugandanenglishnonstandardspeechfinetunev1

This model is a fine-tuned version of facebook/w2v-bert-2.0 on an unknown dataset. It achieves the following results on the evaluation set:

—Loss: 0.7057
—Model Preparation Time: 0.0108
—Wer: 0.3713
—Cer: 0.1728
—Normalized Wer: 0.3283
—Normalized Cer: 0.1626
—Semantic Error: 0.1811

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 5e-06
—trainbatchsize: 2
—evalbatchsize: 2
—seed: 42
—gradientaccumulationsteps: 2
—totaltrainbatch_size: 4
—optimizer: Use OptimizerNames.ADAMWTORCHFUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
—lrschedulertype: cosine
—lrschedulerwarmup_ratio: 0.05
—num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Wer	Cer	Normalized Wer	Normalized Cer	Semantic Error
7.5194	1.0	1293	3.0778	0.0108	0.9999	0.9591	0.9999	0.9567	0.9185
2.6375	2.0	2586	1.9233	0.0108	0.9470	0.5119	0.9443	0.5054	0.8971
1.1816	3.0	3879	1.2664	0.0108	0.7091	0.3115	0.6873	0.3017	0.6103
0.799	4.0	5172	1.0924	0.0108	0.5798	0.2671	0.5483	0.2571	0.5130
0.6397	5.0	6465	1.0465	0.0108	0.5222	0.2427	0.4810	0.2324	0.3658
0.5405	6.0	7758	1.0701	0.0108	0.4742	0.2308	0.4281	0.2206	0.3429
0.4668	7.0	9051	0.9968	0.0108	0.4736	0.2371	0.4300	0.2269	0.3258
0.4103	8.0	10344	1.0442	0.0108	0.4565	0.2244	0.4152	0.2146	0.3213
0.3688	9.0	11637	1.0151	0.0108	0.4605	0.2305	0.4158	0.2209	0.2789
0.3296	10.0	12930	0.9797	0.0108	0.4519	0.2332	0.4072	0.2232	0.2915
0.2962	11.0	14223	1.0263	0.0108	0.4553	0.2284	0.4050	0.2171	0.2708
0.267	12.0	15516	1.1413	0.0108	0.4460	0.2296	0.3945	0.2178	0.2924
0.2406	13.0	16809	1.2067	0.0108	0.4379	0.2277	0.3904	0.2171	0.2726
0.2165	14.0	18102	1.1522	0.0108	0.4317	0.2217	0.3852	0.2113	0.3025

Framework versions

—Transformers 4.56.0
—Pytorch 2.8.0+cu129
—Datasets 2.18.0
—Tokenizers 0.22.0