Modelpublic

KSE-RESEARCH-Group/whisper-medium-dido-yvanchyk-v2

sourceHugging Facemitupdated 8mo agoView on Hugging Face

0likes22downloads

Model Card

whisper-medium - Fine-tuned for Ukrainian ASR

This model is a fine-tuned version of openai/whisper-medium on the Dido Yvanchyk Audio Dataset v2 for Ukrainian speech recognition.

Model Description

Fine-tune openai/whisper-medium on Dido-Yvanchyk dataset

Training Details

Training Data

Property	Value
Dataset	KSE-RESEARCH-Group/Dido-Yvanchyk-Audio-Dataset-v2
Training samples	6811
Validation samples	757
Test samples	842
Language	Ukrainian
Max token length	448

Training Hyperparameters

Parameter	Value
Base model	openai/whisper-medium
Learning rate	1e-05
Warmup steps	500
Max steps	8000
Batch size (per device)	8
Gradient accumulation steps	4
Effective batch size	32
FP16	True
Gradient checkpointing	False
Eval strategy	steps
Eval/Save steps	500
Metric for best model	cer

Training Results

The model was trained for 8000 steps with evaluation every 500 steps. The best checkpoint was selected based on the lowest CER.

Step	Train Loss	Eval Loss	Eval CER (%)	Eval WER (%)
500	0.141	0.226	5.68	16.58
1000	0.051	0.227	4.81	15.41
1500	0.016	0.258	4.53	14.88
2000	0.005	0.284	4.22	14.54
2500	0.005	0.282	4.09	14.16
3000	0.003	0.279	4.12	13.92
3500	0.001	0.3	4.06	14.11
4000	0.001	0.299	4.01	13.83
4500	0.001	0.306	4.1	13.88
5000	0.001	0.311	4.07	13.92
5500	0.0	0.318	4.05	13.92
6000	0.0	0.318	4.03	13.92
6500	0.0	0.322	4.02	13.81
7000	0.0	0.329	3.98	13.7
7500	0.0	0.333	4.07	13.82
8000	0.0	0.334	4.06	13.8

Best Model Checkpoint: Step 7000

Final Evaluation Metrics

Validation Set

Metric	Value
CER	4.06%
WER	13.8%
Eval Loss	0.334

Test Set

Metric	Value
CER	3.96%
WER	14.61%

Usage

Using Pipeline (Recommended)

python

from transformers import pipeline
import torch

device = "cuda:0" if torch.cuda.is_available() else "cpu"

pipe = pipeline(
    "automatic-speech-recognition",
    model="KSE-RESEARCH-Group/whisper-medium-dido-yvanchyk-v2",
    device=device,
)

result = pipe(
    "path/to/audio.wav",
    generate_kwargs={
        "task": "transcribe",
        "language": "ukrainian",
    },
    chunk_length_s=30,
)
print(result["text"])

Using Transformers Directly

python

from transformers import WhisperForConditionalGeneration, WhisperProcessor
import torch

model_id = "KSE-RESEARCH-Group/whisper-medium-dido-yvanchyk-v2"

processor = WhisperProcessor.from_pretrained(model_id)
model = WhisperForConditionalGeneration.from_pretrained(model_id)

# Move to GPU if available
device = "cuda:0" if torch.cuda.is_available() else "cpu"
model = model.to(device)

# Process audio (audio_array should be a numpy array at 16kHz)
input_features = processor(
    audio_array, 
    sampling_rate=16000, 
    return_tensors="pt"
).input_features.to(device)

# Generate transcription
predicted_ids = model.generate(input_features)
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)

Infrastructure

Hardware

Component	Specification
GPU	NVIDIA GeForce RTX 4090
GPU Memory	47.4 GB
GPU Count	1
CUDA Compute Capability	8.9

Environment

Package	Version
Python	3.12.12
PyTorch	2.8.0+cu128
CUDA	12.8
Transformers	4.57.3
Datasets	2.21.0
Evaluate	0.4.6

Training Time

Metric	Value
Total training time	9:19:55.517378
Training started	2025-12-26 09:54:06
Training completed	2025-12-26 19:14:02

Experiment Details

Property	Value
Experiment ID	whisper-medium-002
WandB Project	dido-yvanchik-stt
WandB Run	whisper-medium-002

Citation

If you use this model, please cite:

bibtex

@misc{KSE-RESEARCH-Group-whisper-medium-dido-yvanchyk-v2,
  author = {KSE-RESEARCH-Group},
  title = {whisper-medium - Fine-tuned for Ukrainian ASR},
  year = {2025},
  publisher = {Hugging Face},
  url = {https://huggingface.co/KSE-RESEARCH-Group/whisper-medium-dido-yvanchyk-v2}
}

License

This model is released under the MIT license.

Acknowledgements

—Base model: openai/whisper-medium
—Dataset: Dido Yvanchyk Audio Dataset v2
—Training infrastructure: NVIDIA GeForce RTX 4090