Datasetpublic

playwithmino/mandarin_avsr

Mandarin AVSR Synthetic Mandarin audio-visual speech separation mixes used by byd-avss. Each example is a mixture, a clean target waveform, and a silent grayscale mouth video of the target speaker. Built from Chinese-LiPS, AISHELL-6 Whisper, and WHAM noise. Use of this set must also follow the licenses of those source corpora. Download pip install -U "huggingface_hub[cli]" hf download playwithmino/mandarin_avsr --repo-type dataset --local-dir dataset/mandarin_avsr… See the full description on the dataset page: https://huggingface.co/datasets/playwithmino/mandarin_avsr.

sourceHugging Faceotherupdated 19d agoView on Hugging Face

0likes43downloads

Dataset Card

Mandarin AVSR

Synthetic Mandarin audio-visual speech separation mixes used by byd-avss. Each example is a mixture, a clean target waveform, and a silent grayscale mouth video of the target speaker.

Built from Chinese-LiPS, AISHELL-6 Whisper, and WHAM noise. Use of this set must also follow the licenses of those source corpora.

Download

bash

pip install -U "huggingface_hub[cli]"
hf download playwithmino/mandarin_avsr --repo-type dataset --local-dir dataset/mandarin_avsr
cd dataset/mandarin_avsr
for f in *.tar.gz; do tar -xzf "$f"; done

From the byd-avss repo you can instead run bash scripts/data/download_mandarin_avsr.sh.

Archives: tr_{1-5mix,all}.tar.gz, cv.tar.gz, tt.tar.gz, preview.tar.gz. About 52 GB after extract. Splits: tr (8,000 scenes per 1–5 talker condition), cv and tt (1,000 each). Manifests are in {tr,cv,tt}/all/mix.json and s1.json; media paths in those files are relative to this dataset root.

Layout (after extract)

tr|cv|tt/
  1mix … 5mix/
    audio/mix/*.wav
    audio/s1/*.wav
    mouths/*.npz
    mix.json  s1.json  meta.jsonl
  all/
    mix.json  s1.json