dsenflam/Talker-T2AV-Data

Talker-T2AV-Data

Joint Talking Audio-Video Generation with Autoregressive Diffusion Modeling

Paper (arXiv 2604.23586) · Code (GitHub) · Model · Samples

Clean training data package for Talker-T2AV. Paths in metadata/train.csv are relative to the dataset root after extracting the shard archives.

Contents

• —metadata/train.csv: training index used by Talker-T2AV.
• —shards/*.tar: clean archive shards grouped by modality and dataset source.
• —audio/, motion/, video/: directories created after extracting shards.

train.csv columns:

• —sample_id
• —dataset_source
• —split
• —text
• —wav_path
• —motion_pt_path
• —video_path
• —pt_length
• —fps

Download and Extract

hf download HKUSTAudio/Talker-T2AV-Data --repo-type dataset --local-dir ./Talker-T2AV-Data
cd ./Talker-T2AV-Data
for f in shards/*.tar; do tar -xf "$f"; done

Use With Talker-T2AV Training

export TRAIN_CSV=$(pwd)/metadata/train.csv
export DATA_ROOT=$(pwd)

Talker-T2AV/training/dataset.py resolves wav_path, motion_pt_path, and video_path relative to DATA_ROOT.

Citation

@misc{ye2026talkert2avjointtalkingaudiovideo,
      title={Talker-T2AV: Joint Talking Audio-Video Generation with Autoregressive Diffusion Modeling}, 
      author={Zhen Ye and Xu Tan and Aoxiong Yin and Hongzhan Lin and Guangyan Zhang and Peiwen Sun and Yiming Li and Chi-Min Chan and Wei Ye and Shikun Zhang and Wei Xue},
      year={2026},
      eprint={2604.23586},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2604.23586}, 
}