CoolFace
Datasetpublic

tannhoo06/ViMD_preprocessing

ViMD Truncated 10s — 16kHz Preprocessed version of ViMD (Nguyen et al., EMNLP 2024) for Dialect Identification. Preprocessing applied Resample: 44.1kHz (original) → 16kHz, mono Truncate: only the FIRST 10 SECONDS of each audio are kept (files shorter than 10s are kept intact). 1 original file = 1 sample. This follows the truncation strategy of Lu et al. (2020), NOT chunking. Splits: original ViMD train/valid/test kept unchanged (speaker-exclusive).… See the full description on the dataset page: https://huggingface.co/datasets/tannhoo06/ViMD_preprocessing.

sourceHugging Faceotherupdated 3mo agoView on Hugging Face
0likes14downloads
Dataset Card

ViMD Truncated 10s — 16kHz

Preprocessed version of ViMD (Nguyen et al., EMNLP 2024) for Dialect Identification.

Preprocessing applied

  • —Resample: 44.1kHz (original) → 16kHz, mono
  • —Truncate: only the FIRST 10 SECONDS of each audio are kept (files shorter than 10s are kept intact). 1 original file = 1 sample. This follows the truncation strategy of Lu et al. (2020), NOT chunking.
  • —Splits: original ViMD train/valid/test kept unchanged (speaker-exclusive).

New column vs. original ViMD

  • —new_province: province name after the administrative merger (34 classes), mapped from the original province column (63 classes).

Usage

python
from datasets import load_dataset
ds = load_dataset("tannhoo06/ViMD_preprocessing")

EDA without downloading audio

python
import pandas as pd
meta = pd.read_parquet("hf://datasets/tannhoo06/ViMD_preprocessing/metadata.parquet")

Citation

Please cite the original ViMD paper (Nguyen et al., EMNLP 2024).