Hemantrao/wav2vec2-large-xls-r-300m-hindi_marathi-code-switching-experimentx1

Enhanced Multilingual Code-Switched Speech Recognition for Low-Resource Languages Using Transformer-Based Models and Dynamic Switching Algorithms

Model description

This model is designed to handle code-switched speech in Hindi and Marathi using the wav2vec2-large-xls-r-300m transformer-based model. It leverages advanced techniques such as Q-Learning, SARSA, and Deep Q-Networks (DQN) for determining optimal switch points in code-switched speech.

Intended uses & limitations

Intended uses

• —Automatic speech recognition for multilingual environments involving Hindi and Marathi.
• —Research in multilingual ASR and code-switching phenomena.

Limitations

• —The model may exhibit biases inherent in the training data.
• —Potential limitations in accurately recognizing heavily accented or dialectal speech not covered in the training dataset.

Training params and experimental info

The model was fine-tuned using the following parameters:

• —Attention Dropout: 0.1
• —Hidden Dropout: 0.1
• —Feature Projection Dropout: 0.1
• —Layerdrop: 0.1
• —Learning Rate: 3e-4
• —Mask Time Probability: 0.05

Training dataset

The model was trained on the Common Voice dataset, which includes diverse speech samples in both Hindi and Marathi. The dataset was augmented with synthetically generated code-switched speech to improve the model's robustness in handling code-switching scenarios.

Evaluation results

The model achieved the following performance metrics on the test set:

• —Word Error Rate (WER): 0.2800
• —Character Error Rate (CER): 0.2400