datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
avspeech-visual-audio
AVSpeech Video + Audio
This repository is a media-bearing reconstruction of the public AVSpeech
annotations. Each row represents an already-trimmed segment and keeps the
original source-video timing and target-face-center metadata.
Dataset structure
clip_id: identifier derived as
{youtube_id}_{start_sec:.3f}_{end_sec:.3f}.
avspeech_metadata: JSON containing youtube_id, start_sec, end_sec,
x_center, and y_center from the AVSpeech annotation.
video: video-only… See the full description on the dataset page: https://huggingface.co/datasets/ProgramComputer/avspeech-visual-audio.ASID-1M
ASID-1M: Attribute-Structured and Quality-Verified Audiovisual Instructions
[🏠 Homepage] [📖 Arxiv Paper] [🤗 Models & Datasets] [💻 Code]
Introduction
We introduce ASID-1M, a large-scale audiovisual instruction dataset built to support universal video understanding with fine-grained, controllable supervision.
Most existing video-instruction data represents complex audiovisual content as a single, monolithic caption. This often leads to incomplete coverage (missing audio… See the full description on the dataset page: https://huggingface.co/datasets/AudioVisual-Caption/ASID-1M.audio_visual_starss23_sonAudioVisual-Benchmark-Evaluation
AudioVisual Benchmark Evaluation — evaluation subsets
Item-id lists for the audio-visual benchmark subsets used in our reported
evaluation tables.
Layout
<benchmark>/eval_subset.csv item ids evaluated in the paper
<benchmark>/media_index.csv id -> media filename(s)
<benchmark>/media/ the media files those ids refer to
eval_subset.csv holds a single id column keyed to the source benchmark
(question_id, idx, or index). media/ contains exactly the… See the full description on the dataset page: https://huggingface.co/datasets/plnguyen2908/AudioVisual-Benchmark-Evaluation.visualears-al-32mv2-audio
🗂️ visualears-al-32mv2-audio
English + فارسی · Part of Shenava 1.0 · Project hub · SLT paper submission
🌟 At a glance | معرفی سریع
English
فارسی
🎯 Purpose
32M v2 active-learning audio dataset.
نسخهٔ دوم صف یادگیری فعال مدل ۳۲ میلیونی با انتخابهای بهروزشده.
🧩 Role
active-learning and human-review asset
مصنوع یادگیری فعال و بازبینی انسانی
📦 Snapshot
3,004 files; approximately 443.66 MB
3,004 فایل؛ حدود 443.66 MB
🧱 Packaging
0 Parquet… See the full description on the dataset page: https://huggingface.co/datasets/Reza2kn/visualears-al-32mv2-audio.visualears-al-114m-audio
🗂️ visualears-al-114m-audio
English + فارسی · Part of Shenava 1.0 · Project hub · SLT paper submission
🌟 At a glance | معرفی سریع
English
فارسی
🎯 Purpose
114M active-learning audio dataset.
نمونههای صوتی یادگیری فعال انتخابشده با مدل ۱۱۴ میلیونی شنوا.
🧩 Role
active-learning and human-review asset
مصنوع یادگیری فعال و بازبینی انسانی
📦 Snapshot
3,004 files; approximately 504.53 MB
3,004 فایل؛ حدود 504.53 MB
🧱 Packaging
0 Parquet files and… See the full description on the dataset page: https://huggingface.co/datasets/Reza2kn/visualears-al-114m-audio.visualears-al-32m-audio
🗂️ visualears-al-32m-audio
English + فارسی · Part of Shenava 1.0 · Project hub · SLT paper submission
🌟 At a glance | معرفی سریع
English
فارسی
🎯 Purpose
32M active-learning audio dataset.
نمونههای صوتی یادگیری فعال انتخابشده با مدل ۳۲ میلیونی شنوا.
🧩 Role
active-learning and human-review asset
مصنوع یادگیری فعال و بازبینی انسانی
📦 Snapshot
3,004 files; approximately 447.99 MB
3,004 فایل؛ حدود 447.99 MB
🧱 Packaging
0 Parquet files and 3… See the full description on the dataset page: https://huggingface.co/datasets/Reza2kn/visualears-al-32m-audio.visualears-al-phaseB2-audio
🗂️ visualears-al-phaseB2-audio
English + فارسی · Part of Shenava 1.0 · Project hub · SLT paper submission
🌟 At a glance | معرفی سریع
English
فارسی
🎯 Purpose
Phase B2 active-learning audio dataset.
صف صوتی یادگیری فعال فاز B2 برای بازبینی، اصلاح و ادامهٔ آموزش هدفمند.
🧩 Role
active-learning and human-review asset
مصنوع یادگیری فعال و بازبینی انسانی
📦 Snapshot
4,528 files; approximately 173.42 MB
4,528 فایل؛ حدود 173.42 MB
🧱 Packaging
0… See the full description on the dataset page: https://huggingface.co/datasets/Reza2kn/visualears-al-phaseB2-audio.audiovisual_translation_dub
Audiovisual Translation Dubbing Dataset
A paired video dataset for training video dubbing and lip-sync models, as described in the paper "JUST-DUB-IT: Video Dubbing via Joint Audio-Visual Diffusion".
📄 Project: justdubit.github.io💻 Code: github.com/justdubit/just-dub-it🤗 Model: huggingface.co/justdubit/justdubit
Dataset Description
This dataset contains paired video samples for training synchronized audio-video dubbing models. Each sample includes:
Target video:… See the full description on the dataset page: https://huggingface.co/datasets/justdubit/audiovisual_translation_dub.visualears-uncertain-review-audio
🗂️ visualears-uncertain-review-audio
English + فارسی · Part of Shenava 1.0 · Project hub · SLT paper submission
🌟 At a glance | معرفی سریع
English
فارسی
🎯 Purpose
Uncertain-review active-learning audio queue.
صف بازبینی نمونههای نامطمئن برای تشخیص خطا، اصلاح برچسب و جلوگیری از ورود دادهٔ کماعتماد.
🧩 Role
active-learning and human-review asset
مصنوع یادگیری فعال و بازبینی انسانی
📦 Snapshot
348 files; approximately 35.98 MB
348 فایل؛ حدود… See the full description on the dataset page: https://huggingface.co/datasets/Reza2kn/visualears-uncertain-review-audio.visualears-nonpersian-review-audio
🗂️ visualears-nonpersian-review-audio
English + فارسی · Part of Shenava 1.0 · Project hub · SLT paper submission
🌟 At a glance | معرفی سریع
English
فارسی
🎯 Purpose
Non-Persian review/filtering audio dataset.
صف کنترل زبان برای شناسایی و جداسازی صوتهای غیرفارسی یا مشکوک.
🧩 Role
active-learning and human-review asset
مصنوع یادگیری فعال و بازبینی انسانی
📦 Snapshot
528 files; approximately 31.70 MB
528 فایل؛ حدود 31.70 MB
🧱 Packaging
0… See the full description on the dataset page: https://huggingface.co/datasets/Reza2kn/visualears-nonpersian-review-audio.visualears-cutverify-audio
🗂️ visualears-cutverify-audio
English + فارسی · Part of Shenava 1.0 · Project hub · SLT paper submission
🌟 At a glance | معرفی سریع
English
فارسی
🎯 Purpose
Cut verification audio dataset.
نمونههای صوتی ویژهٔ کنترل مرز برش، کاملبودن گفتار و همترازی قطعه با متن.
🧩 Role
active-learning and human-review asset
مصنوع یادگیری فعال و بازبینی انسانی
📦 Snapshot
603 files; approximately 35.82 MB
603 فایل؛ حدود 35.82 MB
🧱 Packaging
0 Parquet files… See the full description on the dataset page: https://huggingface.co/datasets/Reza2kn/visualears-cutverify-audio.AudioVisualMouseSaliency
ECCV-AIM Video Saliency Prediction Challenge 2024
Dataset
We provide a novel audio-visual mouse saliency (AViMoS) dataset with the following key-features:
Diverse content: movie, sports, live, vertical videos, etc.;
Large scale: 1500 videos with mean 19s duration;
High resolution: all streams are FullHD;
Audio track saved and played to observers;Mouse fixations from >5000 observers (>70 per video);
License: CC-BY;
File structure:
Videos.zip — 1500 (1000 Train +… See the full description on the dataset page: https://huggingface.co/datasets/ANDRYHA/AudioVisualMouseSaliency.visualears-active-learning-669-audio
🗂️ visualears-active-learning-669-audio
English + فارسی · Part of Shenava 1.0 · Project hub · SLT paper submission
🌟 At a glance | معرفی سریع
English
فارسی
🎯 Purpose
Active-learning correction audio dataset.
صوتهای انتخابشده برای اصلاح فعال؛ مناسب بازبینی انسانی و افزودن نمونههای آموزنده به چرخهٔ آموزش.
🧩 Role
active-learning and human-review asset
مصنوع یادگیری فعال و بازبینی انسانی
📦 Snapshot
671 files; approximately 11.13 MB
671 فایل؛… See the full description on the dataset page: https://huggingface.co/datasets/Reza2kn/visualears-active-learning-669-audio.visualears-telephooney-calibration-audio
🗂️ visualears-telephooney-calibration-audio
English + فارسی · Part of Shenava 1.0 · Project hub · SLT paper submission
🌟 At a glance | معرفی سریع
English
فارسی
🎯 Purpose
Telephooney calibration audio dataset.
دادهٔ کالیبراسیون گفتار تلفنی برای تشخیص کانال باریکباند و شرایط مخابراتی.
🧩 Role
quality calibration and data-selection asset
مصنوع کالیبراسیون کیفیت و انتخاب داده
📦 Snapshot
172 files; approximately 20.50 MB
172 فایل؛ حدود 20.50 MB… See the full description on the dataset page: https://huggingface.co/datasets/Reza2kn/visualears-telephooney-calibration-audio.visualears-crap-calibration-audio
🗂️ visualears-crap-calibration-audio
English + فارسی · Part of Shenava 1.0 · Project hub · SLT paper submission
🌟 At a glance | معرفی سریع
English
فارسی
🎯 Purpose
Data-quality / crap-detector calibration audio dataset.
دادهٔ کالیبراسیون کیفیت برای آموزش و ارزیابی آشکارساز نمونههای خراب یا نامناسب.
🧩 Role
quality calibration and data-selection asset
مصنوع کالیبراسیون کیفیت و انتخاب داده
📦 Snapshot
183 files; approximately 10.44 MB
183 فایل؛… See the full description on the dataset page: https://huggingface.co/datasets/Reza2kn/visualears-crap-calibration-audio.visualears-fa-train-audio-16k
VisualEars 115M FA training audio (16kHz mono FLAC)
3362186 clips packed into 89 tar shards (~5GB each), + NeMo manifests.
Fast download + extract on a new cluster
pip install -U huggingface_hub
hf download Reza2kn/visualears-fa-train-audio-16k --repo-type dataset --local-dir DATA # add: --token $HF_TOKEN if private
cd DATA && for t in audio/shard_*.tar; do tar xf "$t"; done # reconstructs pseudo_audio/, gold_*_audio/
Manifests in manifests/ use paths… See the full description on the dataset page: https://huggingface.co/datasets/Reza2kn/visualears-fa-train-audio-16k.audio-visual-convo
Audio-Visual (AV) Convo
Dataset Description
Audio-Visual (AV) Convo is a multimodal dataset of natural conversations between pairs of friends, captured with two simultaneous camera angles and a separate audio recording. Each sample is a side-by-side video combining both camera views with the original audio track.
Key Characteristics
Conversations between friends talking about casual/life topics
Diverse demographics: participants range in age… See the full description on the dataset page: https://huggingface.co/datasets/liva-ai/audio-visual-convo.Audio-Visual-Speech-Recognition-VIAudio_Visual_Understandingvisual-audio-opensource