Modelpublic

csikasote/mms-1b-all-bemgen-combined-train-drodat-gdro-1.65e-4-dat-0.019-no-sd-0.00-42

sourceHugging Facecc-by-nc-4.0updated 6d agoView on Hugging Face

0likes110downloads

Model Card

mms-1b-all-bemgen-combined-train-drodat-gdro-1.65e-4-dat-0.019-no-sd-0.00-42

This model is a fine-tuned version of facebook/mms-1b-all on the BEMGEN - BEM dataset. It achieves the following results on the evaluation set:

—Loss: 12.2430
—Cer: 0.0530

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 0.0003
—trainbatchsize: 4
—evalbatchsize: 4
—seed: 42
—gradientaccumulationsteps: 2
—totaltrainbatch_size: 8
—optimizer: Use OptimizerNames.ADAMWTORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizerargs=No additional optimizer arguments
—lrschedulertype: linear
—lrschedulerwarmup_steps: 500
—num_epochs: 30.0
—mixedprecisiontraining: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Cer
268.2696	0.6349	500	264.2914	0.9222
30.1189	1.2692	1000	14.7193	0.0653
30.1291	1.9041	1500	13.8787	0.0617
27.8786	2.5384	2000	13.6126	0.0603
24.8185	3.1727	2500	13.6259	0.0594
25.3736	3.8076	3000	13.1907	0.0581
24.9214	4.4419	3500	13.3854	0.0569
25.3807	5.0762	4000	12.8844	0.0563
22.826	5.7111	4500	12.7335	0.0559
21.8196	6.3454	5000	12.8145	0.0554
21.9892	6.9803	5500	12.8158	0.0555
20.2641	7.6146	6000	12.5432	0.0544
21.3521	8.2489	6500	12.7404	0.0548
19.4366	8.8838	7000	12.5614	0.0543
20.0923	9.5181	7500	12.2429	0.0530
18.2857	10.1524	8000	12.4929	0.0538
19.1447	10.7873	8500	12.2578	0.0525
18.0904	11.4216	9000	12.5402	0.0530

Framework versions

—Transformers 4.53.0
—Pytorch 2.10.0+cu128
—Datasets 4.5.0
—Tokenizers 0.21.4