datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
avspeech-visual-audio
AVSpeech Video + Audio
This repository is a media-bearing reconstruction of the public AVSpeech
annotations. Each row represents an already-trimmed segment and keeps the
original source-video timing and target-face-center metadata.
Dataset structure
clip_id: identifier derived as
{youtube_id}_{start_sec:.3f}_{end_sec:.3f}.
avspeech_metadata: JSON containing youtube_id, start_sec, end_sec,
x_center, and y_center from the AVSpeech annotation.
video: video-only… See the full description on the dataset page: https://huggingface.co/datasets/ProgramComputer/avspeech-visual-audio.ASID-1M
ASID-1M: Attribute-Structured and Quality-Verified Audiovisual Instructions
[🏠 Homepage] [📖 Arxiv Paper] [🤗 Models & Datasets] [💻 Code]
Introduction
We introduce ASID-1M, a large-scale audiovisual instruction dataset built to support universal video understanding with fine-grained, controllable supervision.
Most existing video-instruction data represents complex audiovisual content as a single, monolithic caption. This often leads to incomplete coverage (missing audio… See the full description on the dataset page: https://huggingface.co/datasets/AudioVisual-Caption/ASID-1M.AudioVisual-Benchmark-Evaluation
AudioVisual Benchmark Evaluation — evaluation subsets
Item-id lists for the audio-visual benchmark subsets used in our reported
evaluation tables.
Layout
<benchmark>/eval_subset.csv item ids evaluated in the paper
<benchmark>/media_index.csv id -> media filename(s)
<benchmark>/media/ the media files those ids refer to
eval_subset.csv holds a single id column keyed to the source benchmark
(question_id, idx, or index). media/ contains exactly the… See the full description on the dataset page: https://huggingface.co/datasets/plnguyen2908/AudioVisual-Benchmark-Evaluation.visualears-fa-train-audio-16k
VisualEars 115M FA training audio (16kHz mono FLAC)
3362186 clips packed into 89 tar shards (~5GB each), + NeMo manifests.
Fast download + extract on a new cluster
pip install -U huggingface_hub
hf download Reza2kn/visualears-fa-train-audio-16k --repo-type dataset --local-dir DATA # add: --token $HF_TOKEN if private
cd DATA && for t in audio/shard_*.tar; do tar xf "$t"; done # reconstructs pseudo_audio/, gold_*_audio/
Manifests in manifests/ use paths… See the full description on the dataset page: https://huggingface.co/datasets/Reza2kn/visualears-fa-train-audio-16k.audio-visual-convo
Audio-Visual (AV) Convo
Dataset Description
Audio-Visual (AV) Convo is a multimodal dataset of natural conversations between pairs of friends, captured with two simultaneous camera angles and a separate audio recording. Each sample is a side-by-side video combining both camera views with the original audio track.
Key Characteristics
Conversations between friends talking about casual/life topics
Diverse demographics: participants range in age… See the full description on the dataset page: https://huggingface.co/datasets/liva-ai/audio-visual-convo.Audio-Visual-Speech-Recognition-VI