datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
infore2_audiobooks
unofficial mirror of InfoRe Technology public dataset №2
official announcement: https://www.facebook.com/groups/j2team.community/permalink/1010834009248719/
415h, 315k samples, vietnamese audiobooks of chinese wǔxiá 武俠 & xiānxiá 仙俠
bộ dữ liệu bóc ra từ YouTube đọc truyện võ hiệp & tiên hiệp, áp dụng kĩ thuật đối chiếu văn bản để dán nhãn tự động
official download:… See the full description on the dataset page: https://huggingface.co/datasets/doof-ferb/infore2_audiobooks.vlsp2020_vinai_100h
unofficial mirror of VLSP 2020 - VinAI - ASR challenge dataset
official announcement:
tiếng việt: https://institute.vinbigdata.org/events/vinbigdata-chia-se-100-gio-du-lieu-tieng-noi-cho-cong-dong/
in eglish: https://institute.vinbigdata.org/en/events/vinbigdata-shares-100-hour-data-for-the-community/
VLSP 2020 workshop: https://vlsp.org.vn/vlsp2020
official download: https://drive.google.com/file/d/1vUSxdORDxk-ePUt-bUVDahpoXiqKchMx/view?usp=sharing
contact: info@vinbigdata.org… See the full description on the dataset page: https://huggingface.co/datasets/doof-ferb/vlsp2020_vinai_100h.LSVSC
unofficial mirror of LSVSC dataset (novel large-scale Vietnamese speech corpus)
official announcement: https://www.mdpi.com/2079-9292/13/5/977
official download: https://drive.google.com/drive/folders/1tiPKaIOC7bt6isv5qFqf61O_2jFK8ZOI
100h, 57k samples
pre-process: see my code: https://github.com/phineas-pta/fine-tune-whisper-vi/blob/main/misc/clean-lsvsc.py
need to do: check misspelling, restore foreign words phonetised to vietnamese
usage with HuggingFace:
# pip install -q… See the full description on the dataset page: https://huggingface.co/datasets/doof-ferb/LSVSC.infore1_25hours
unofficial mirror of InfoRe Technology public dataset №1
official announcement: https://www.facebook.com/groups/j2team.community/permalink/1010834009248719/
25h, 14.9k samples, InfoRe paid a contractor to read text
official download: magnet:?xt=urn:btih:1cbe13fb14a390c852c016a924b4a5e879d85f41&dn=25hours.zip&tr=http%3A%2F%2Foffice.socials.vn%3A8725%2Fannounce
mirror: https://files.huylenguyen.com/datasets/infore/25hours.zip
unzip password: BroughtToYouByInfoRe
pre-process: see… See the full description on the dataset page: https://huggingface.co/datasets/doof-ferb/infore1_25hours.VietMed_unlabeled
unofficial mirror of VietMed (Vietnamese speech data in medical domain) unlabeled set
official announcement: https://arxiv.org/abs/2404.05659
official download: https://huggingface.co/datasets/leduckhai/VietMed
this repo contains the unlabeled set: 966h - 230k samples
i also gather the metadata: see info.csv
my extraction code: https://github.com/phineas-pta/fine-tune-whisper-vi/blob/main/misc/vietmed-unlabeled.py
need to do: check misspelling, restore foreign words phonetised to… See the full description on the dataset page: https://huggingface.co/datasets/doof-ferb/VietMed_unlabeled.fpt_fosd
unofficial mirror of FPT Open Speech Dataset (FOSD)
released publicly in 2018 by FPT Corporation
100h, 25.9k samples
official link (dead): https://fpt.ai/fpt-open-speech-data/
mirror: https://data.mendeley.com/datasets/k9sxg2twv4/4
DOI: 10.17632/k9sxg2twv4.4
pre-process:
remove non-sense strings: -N \r\n
remove 4 files because missing transcription:
Set001_V0.1_008210.mp3
Set001_V0.1_010753.mp3
Set001_V0.1_011477.mp3
Set001_V0.1_011841.mp3
need to do: check misspelling
usage… See the full description on the dataset page: https://huggingface.co/datasets/doof-ferb/fpt_fosd.VietMed_labeled
unofficial mirror of VietMed (Vietnamese speech data in medical domain) labeled set
official announcement: https://arxiv.org/abs/2404.05659
official download: https://huggingface.co/datasets/leduckhai/VietMed
this repo contains the labeled set: 9.2k samples
i also gather the metadata: see info.csv
my extraction code: https://github.com/phineas-pta/fine-tune-whisper-vi/blob/main/misc/vietmed-labeled.py
need to do: check misspelling, restore foreign words phonetised to vietnamese… See the full description on the dataset page: https://huggingface.co/datasets/doof-ferb/VietMed_labeled.vais1000
unofficial mirror of VAIS-1000
official announcement: https://vais.vn/vi/tai-ve/hts_for_vietnamese (dead)
mirror: https://github.com/undertheseanlp/text_to_speech/tree/run/data/vais1000/raw
small only 1h40min audio - 1 speaker (female northern accent) - 1k samples
pre-process: none
need to do: check misspelling, restore foreign words phonetised to vietnamese
usage with HuggingFace:
# pip install -q "datasets[audio]"
from datasets import load_dataset
from torch.utils.data import… See the full description on the dataset page: https://huggingface.co/datasets/doof-ferb/vais1000.VietMDD
unofficial mirror of VietMMD (Mispronunciation Detection and Diagnosis)
official announcement: https://github.com/VietMDDDataset/VietMDD
official download: https://drive.google.com/drive/folders/1TjTluTxEB99QhGFTYFWb-vEdWXM-lyKJ?usp=sharing
DOI: 10.21437/Interspeech.2023-364
5h, 4.2k samples
pre-process: see my code: https://github.com/phineas-pta/fine-tune-whisper-vi/blob/main/misc/viet-mdd.py
custom split: orphan: speech without any transcription unlike in train/validation/test… See the full description on the dataset page: https://huggingface.co/datasets/doof-ferb/VietMDD.Speech-MASSIVE_vie
Vietnamse subset of the Speech-MASSIVE dataset
extracted from:
https://huggingface.co/datasets/FBK-MT/Speech-MASSIVE
https://huggingface.co/datasets/FBK-MT/Speech-MASSIVE-test
my extraction code: https://github.com/phineas-pta/fine-tune-whisper-vi/blob/main/misc/load-speechmassive.py
BibleMMS_vie
Vietnamse subset of the BibleMMS dataset
extracted from: https://huggingface.co/datasets/Flux9665/BibleMMS
my extraction code: https://github.com/phineas-pta/fine-tune-whisper-vi/blob/main/misc/load-biblemms.py
gigaspeech2_vie
Vietnamse subset of the Gigaspeech2 dataset
extracted from: https://huggingface.co/datasets/speechcolab/gigaspeech2