Modelpublic

csikasote/mms-1b-all-bemgen-combined-train-droctc-gdro-3e-2-dat-5e-1-no-sd-0.00-42

sourceHugging Facecc-by-nc-4.0updated 6d agoView on Hugging Face

0likes212downloads

Model Card

mms-1b-all-bemgen-combined-train-droctc-gdro-3e-2-dat-5e-1-no-sd-0.00-42

This model is a fine-tuned version of facebook/mms-1b-all on the BEMGEN - BEM dataset. It achieves the following results on the evaluation set:

—Loss: 12.2996
—Cer: 0.0527

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 0.0003
—trainbatchsize: 8
—evalbatchsize: 4
—seed: 42
—gradientaccumulationsteps: 2
—totaltrainbatch_size: 16
—optimizer: Use OptimizerNames.ADAMWTORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizerargs=No additional optimizer arguments
—lrschedulertype: linear
—lrschedulerwarmup_steps: 500
—num_epochs: 30.0
—mixedprecisiontraining: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Cer
286.8689	1.2690	500	247.0006	0.9359
36.1192	2.5381	1000	14.3835	0.0626
30.9824	3.8071	1500	13.7055	0.0603
30.3313	5.0761	2000	13.4051	0.0585
29.8046	6.3452	2500	13.0660	0.0573
27.5677	7.6142	3000	12.8192	0.0563
25.4127	8.8832	3500	12.9092	0.0570
26.9972	10.1523	4000	12.8406	0.0574
24.4469	11.4213	4500	12.6555	0.0554
25.0045	12.6904	5000	12.6532	0.0549
23.3332	13.9594	5500	12.6980	0.0552
22.4817	15.2284	6000	12.6626	0.0545
23.2254	16.4975	6500	12.5046	0.0540
24.0542	17.7665	7000	12.4857	0.0545
23.0318	19.0355	7500	12.3897	0.0530
20.7927	20.3046	8000	12.3000	0.0527
19.9905	21.5736	8500	12.3507	0.0537
20.5534	22.8426	9000	12.3648	0.0528
19.0877	24.1117	9500	12.3617	0.0533

Framework versions

—Transformers 4.53.0
—Pytorch 2.10.0+cu128
—Datasets 4.5.0
—Tokenizers 0.21.4