MatsRooth/wav2vec2-base_than_I_did

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

wav2vec2-basethanI_did

This model is a fine-tuned version of facebook/wav2vec2-base on the MatsRooth/thanIdid dataset. It achieves the following results on the evaluation set:

• —Loss: 0.2077
• —Accuracy: 0.9592

Model description

This is a binary classifier for the prosody of tokens of "I did". The label s is subject prominence. The label ns is the complement, with prominence either on "did" or afterwards.

Intended uses & limitations

Research on prosody.

Training and evaluation data

The utterances are collected on Youtube, aligned with the Youtube transcript using Kaldi, and cut to the words "I did" using Matlab. Labels were assigned by the experimenter, using 's' for tokens there the main clause subject differed from the than-clause subject, and 'ns' for other tokens. The labeling does not depend on prosody, though it correlates with it.

On the same problem using an SVM classifier, see Howell, Jonathan, Mats Rooth, and Michael Wagner, Acoustic classification of focus: On the web and in the lab (2016).

The class ns was reduced to 160 tokens, to match the number of tokens of s.

Training procedure

Training and evaluation use runaudioclassification.py from HuggingFace. The slurm script thanIdid.sub launches training.

Training hyperparameters

The following hyperparameters were used during training:

• —learning_rate: 3e-05
• —trainbatchsize: 16
• —evalbatchsize: 16
• —seed: 0
• —gradientaccumulationsteps: 2
• —totaltrainbatch_size: 32
• —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• —lrschedulertype: linear
• —lrschedulerwarmup_ratio: 0.1
• —num_epochs: 20.0
• —mixedprecisiontraining: Native AMP

Training results

Framework versions

• —Transformers 4.36.0.dev0
• —Pytorch 2.1.0+cu121
• —Datasets 2.13.1
• —Tokenizers 0.15.0