Modelpublic

qthuan2604/ViT5_Restore_Diacritics_Vietnamese

sourceHugging Facemitupdated 11mo agoView on Hugging Face

0likes28downloads

Model Card

ViT5RestoreDiacritics_Vietnamese

This model is a fine-tuned version of VietAI/vit5-base on an unknown dataset. It achieves the following results on the evaluation set:

—Loss: 0.0897
—Model Preparation Time: 0.0058
—Cer: 0.2766
—Wer: 0.3313
—Char Acc: 0.6266

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 1e-05
—trainbatchsize: 12
—evalbatchsize: 48
—seed: 42
—gradientaccumulationsteps: 4
—totaltrainbatch_size: 48
—optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
—lrschedulertype: cosine
—lrschedulerwarmup_ratio: 0.1
—num_epochs: 10
—mixedprecisiontraining: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Cer	Wer	Char Acc
5.3339	0.9985	333	0.3909	0.0058	0.5055	0.7208	0.2451
0.3762	2.0	667	0.2121	0.0058	0.3642	0.4929	0.4239
0.2409	2.9985	1000	0.1510	0.0058	0.3203	0.4134	0.5124
0.1789	4.0	1334	0.1212	0.0058	0.3018	0.3760	0.5581
0.1451	4.9985	1667	0.1054	0.0058	0.2890	0.3537	0.5900
0.1247	6.0	2001	0.0974	0.0058	0.2827	0.3434	0.6097
0.1137	6.9985	2334	0.0931	0.0058	0.2793	0.3369	0.6196
0.1064	8.0	2668	0.0906	0.0058	0.2774	0.3328	0.6244
0.103	8.9985	3001	0.0898	0.0058	0.2769	0.3317	0.6255
0.1016	9.9850	3330	0.0897	0.0058	0.2766	0.3313	0.6266

Framework versions

—Transformers 4.43.3
—Pytorch 2.6.0+cu124
—Datasets 3.1.0
—Tokenizers 0.19.1