classla/wav2vec2-xls-r-juznevesti-sr
1159k
wav2vec2-xls-r-juznevesti
This model for Serbian ASR is based on the facebook/wav2vec2-xls-r-300m model and was fine-tuned with 58 hours of audio and transcripts from Južne vesti, programme '15 minuta'.
For more info on the dataset creation see this repo.
Metrics
Evaluation is performed on the dev and test portions of the JuzneVesti dataset
Usage in transformers
Tested with transformers==4.18.0, torch==1.11.0, and SoundFile==0.10.3.post1.
from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC
import soundfile as sf
import torch
import os
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
# load model and tokenizer
processor = Wav2Vec2Processor.from_pretrained(
"classla/wav2vec2-xls-r-juznevesti-sr")
model = Wav2Vec2ForCTC.from_pretrained("classla/wav2vec2-xls-r-juznevesti-sr")
# download the example wav files:
os.system("wget https://huggingface.co/classla/wav2vec2-xls-r-parlaspeech-hr/raw/main/00020570a.flac.wav")
# read the wav file
speech, sample_rate = sf.read("00020570a.flac.wav")
input_values = processor(speech, sampling_rate=sample_rate, return_tensors="pt").input_values.to(device)
# remove the raw wav file
os.system("rm 00020570a.flac.wav")
# retrieve logits
logits = model.to(device)(input_values).logits
# take argmax and decode
predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.decode(predicted_ids[0])
transcription # 'velik broj poslovnih subjekata posluje sa minosom velik deo'Training hyperparameters
In fine-tuning, the following arguments were used: