Modelpublic

suideepmax/wav2vec2-large-960h-lv60-self-atc-uwb-atcc

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face

0likes4downloads

Model Card

0.0ld0.0ad0.0attd0.0fpd0.01mtp12mtl0.0mfp12mfl16acc

This model is a fine-tuned version of facebook/wav2vec2-large-960h-lv60-self on the EXPERIMENTS/DATA/UWB_ATCC/TRAIN - NA dataset. It achieves the following results on the evaluation set:

—Loss: 0.9386
—Wer: 0.1507

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 0.0005
—trainbatchsize: 1
—evalbatchsize: 12
—seed: 42
—distributed_type: multi-GPU
—num_devices: 4
—gradientaccumulationsteps: 16
—totaltrainbatch_size: 64
—totalevalbatch_size: 48
—optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
—lrschedulertype: linear
—lrschedulerwarmup_steps: 1000
—training_steps: 10000
—mixedprecisiontraining: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
No log	2.78	500	0.6754	0.2799
1.9717	5.56	1000	0.5679	0.2050
1.9717	8.33	1500	0.5365	0.1873
0.6024	11.11	2000	0.5301	0.1898
0.6024	13.89	2500	0.5311	0.1736
0.4164	16.67	3000	0.5608	0.1726
0.4164	19.44	3500	0.6315	0.1777
0.3101	22.22	4000	0.6357	0.1773
0.3101	25.0	4500	0.6805	0.1623
0.2305	27.78	5000	0.7486	0.1639
0.2305	30.56	5500	0.7124	0.1620
0.1764	33.33	6000	0.7042	0.1587
0.1764	36.11	6500	0.7720	0.1628
0.1333	38.89	7000	0.8483	0.1580
0.1333	41.67	7500	0.8459	0.1624
0.0996	44.44	8000	0.8556	0.1547
0.0996	47.22	8500	0.9197	0.1545
0.076	50.0	9000	0.9336	0.1505
0.076	52.78	9500	0.9272	0.1509
0.0606	55.56	10000	0.9386	0.1507

Framework versions

—Transformers 4.24.0
—Pytorch 1.13.0+cu117
—Datasets 2.14.0
—Tokenizers 0.13.3