datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Clotho-Moment
Clotho-Moment
This repository provides wav files used in Language-based Audio Moment Retrieval.
Each sample includes long audio containing some audio events with the temporal and textual annotation.
Project page: https://h-munakata.github.io/Language-based-Audio-Moment-Retrieval/
Code: https://github.com/line/lighthouse
Split
Train
train/train-{000..715}.tar
37930 audio samples
Valid
valid/valid-{000..108}.tar
5741 audio samples
Test
test/test-{000..142}.tar
7569… See the full description on the dataset page: https://huggingface.co/datasets/lighthouse-emnlp2024/Clotho-Moment.vibench
VIBench
VIBench is a benchmark for measuring vertical integration bias in
direct and agentic code generation. It contains 20 direct scenarios and
20 aligned agentic workflows covering realistic software integration
choices across cloud and API ecosystems. The bundle includes the
benchmark tasks, model metadata, system prompts, provider-proof
artifacts, and the blind detector-audit sample used for validation.
Viewer splits
The dataset viewer is intentionally simplified to… See the full description on the dataset page: https://huggingface.co/datasets/vibench-emnlp26/vibench.emnlp-2026-ifr-mair-full
EMNLP 2026 Findings: On-Policy Distillation Meets Off-Policy GRPO — Training Compact Instruction-Following Rerankers
Full MAIR set used for evaluation — 126 tasks and 9,356 queries (~8.8M candidate documents), spanning heterogeneous instruction-following retrieval domains.
Overview
This dataset is the complete MAIR (Massive Instructed Retrieval Benchmark) collection used as the out-of-distribution evaluation suite in the paper. It bundles all 126 MAIR tasks in a… See the full description on the dataset page: https://huggingface.co/datasets/anonymousauthor01/emnlp-2026-ifr-mair-full.vibench-results
VIBench Results
VIBench Results contains the retained raw generations,
detector-labeled outputs, complete runs, option-order and runtime
ablations, paper-facing summaries, figures, configs, audit files, and
static explorer indices used in the study.
The main paper evaluation covers 13 models, 15,600 direct generations,
and 2,000 agentic runs. Direct and agentic VIB are reported as
scenario-matched, share-normalized differences in affiliated-ecosystem
selection relative to strict… See the full description on the dataset page: https://huggingface.co/datasets/vibench-emnlp26/vibench-results.EMNLP_Cost-Aware-Protocol-Routing
Cost-Aware Protocol Routing: Matched Protocol Outcomes
The short version. We ran the same 6,803 reasoning problems through four
different LLM collaboration setups — from a single direct answer up to a
four-agent deliberation — and recorded, for every problem, which ones got it
right. Then we asked whether a model can look at a problem beforehand and
predict which setup is worth paying for.
It can predict whether it will fail. It cannot predict which collaboration
protocol will… See the full description on the dataset page: https://huggingface.co/datasets/AgentsSci/EMNLP_Cost-Aware-Protocol-Routing.uncertainty-vlm-llama-emnlp_stage
uncertainty-vlm-llama-official EMNLP stage-wise train/dev/test splits
This dataset contains deterministic stratified 80/10/10 train/dev/test assignments for the four diagram stages. Each split uses the stage_label column as the binary target for that stage.
uncertainty-vlm-gemma-emnlp_stage
uncertainty-vlm-gemma-official EMNLP stage-wise train/dev/test splits
This dataset contains deterministic stratified 80/10/10 train/dev/test assignments for the four diagram stages. Each split uses the stage_label column as the binary target for that stage.
emnlp2026_dataset
When Synthetic Data Hurts: On Catastrophic Forgetting in Skill Retrieval for LLM Agents
This Readme file serves as a guide for using the two datasets we presented in our paper: Harbor Trial dataset which consists of real task execution data and Track B dataset which contains synthetic data. Those datasets are intended to finetune your own skill retrieval solutions and recipes and serve as a common ground to compare new catastrophic forgetting mitigation methodologies with our… See the full description on the dataset page: https://huggingface.co/datasets/manulife/emnlp2026_dataset.emnlp-2020-2025-atomic-claims
EMNLP 2020–2025 Atomic Contribution Claims (ACC), with drift clusters
18,293 atomic contribution claims extracted from the abstracts of the full EMNLP main track 2020–2025, plus the canonical 80-cluster drift clustering and per-cluster drift statistics used in the Drift Inspector paper.
An atomic contribution claim (ACC) is a single self-contained sentence stating one concrete contribution of a paper: atomic (one contribution-bearing proposition), decontextualized (pronouns… See the full description on the dataset page: https://huggingface.co/datasets/Hamyrappy/emnlp-2020-2025-atomic-claims.uncertainty-vlm-qwen3-emnlp_stage
uncertainty-vlm-qwen3-official EMNLP stage-wise train/dev/test splits
This dataset contains deterministic stratified 80/10/10 train/dev/test assignments for the four diagram stages. Each split uses the stage_label column as the binary target for that stage.
uncertainty-vlm-qwen2p5-emnlp_stage
uncertainty-vlm-qwen2p5-official EMNLP stage-wise train/dev/test splits
This dataset contains deterministic stratified 80/10/10 train/dev/test assignments for the four diagram stages. Each split uses the stage_label column as the binary target for that stage.
VIVA_Benchmark_EMNLP24
VIVA: A Benchmark for Vision-Grounded Decision-Making with Human Values
Zhe Hu1,
Yixiao Ren1,
Jing Li1,
Yu Yin2
1The Hong Kong Polytechnic University
2Case Western Reserve University
EMNLP 2024 (main)
📄 Paper
🌎 Website
💻 Code
Introduction
This is the official huggingface repo providing the benchmark of our EMNLP'24 paper:
VIVA: A Benchmark for… See the full description on the dataset page: https://huggingface.co/datasets/zhehuderek/VIVA_Benchmark_EMNLP24.CASTELLA
CASTELLA
This repository provides wav files used in CASTELLA: Long Audio Dataset with Captions and Temporal Boundaries
. This dataset is originally provided in GitHub.
Each sample includes long audio containing some audio events with the temporal and textual annotation.
Project page: https://h-munakata.github.io/CASTELLA-demo/
Code: https://github.com/line/lighthouse
This repository only contains annotation data, not audio data.
Extracted features are available on HF.
If you need… See the full description on the dataset page: https://huggingface.co/datasets/lighthouse-emnlp2024/CASTELLA.EMNLP-Accepted-PapersCASTELLA_CLAP_features
CASTELLA CLAP features
This repository contains audio and text features of CASTELLA dataset extracted by CLAP.
Using these features, we can reproduce the audio moments retrieval using CASTELLA, which is used in lighthouse.
Please also check demo page.
How to Download?
Run the following script:
from huggingface_hub import snapshot_download
repo_id = "lighthouse-emnlp2024/CASTELLA_CLAP_features"
local_dir = "./"
downloaded_path = snapshot_download(… See the full description on the dataset page: https://huggingface.co/datasets/lighthouse-emnlp2024/CASTELLA_CLAP_features.EMNLP2024uncertainty-vlm-llama-emnlp_test
uncertainty-vlm-llama-official EMNLP stage-wise test splits
This dataset contains only the test split from deterministic stratified
80/10/10 train/dev/test assignments for the four diagram stages.
emnlp-2026-ifr-train-val-set
EMNLP 2026 Findings: On-Policy Distillation Meets Off-Policy GRPO — Training Compact Instruction-Following Rerankers
Train and validation splits used to train and evaluate the instruction-following reranker (IFR) in an in-distribution setting. The benchmark is an aggregated compilation of eight public instruction-following retrieval datasets spanning web search, code, mathematics, news, and multi-hop retrieval.
Composition
Source dataset
Train
Validation… See the full description on the dataset page: https://huggingface.co/datasets/anonymousauthor01/emnlp-2026-ifr-train-val-set.uncertainty-vlm-qwen3-emnlp_test
uncertainty-vlm-qwen3-official EMNLP stage-wise test splits
This dataset contains only the test split from deterministic stratified
80/10/10 train/dev/test assignments for the four diagram stages.
uncertainty-vlm-gemma-emnlp_test
uncertainty-vlm-gemma-official EMNLP stage-wise test splits
This dataset contains only the test split from deterministic stratified
80/10/10 train/dev/test assignments for the four diagram stages.
EMNLP2025uncertainty-vlm-qwen2p5-emnlp_test
uncertainty-vlm-qwen2p5-official EMNLP stage-wise test splits
This dataset contains only the test split from deterministic stratified
80/10/10 train/dev/test assignments for the four diagram stages.
uncertainty-vlm-llama-emnlpEMNLP2021EMNLP2022RaVE_emnlp23
Dataset Card for VECHR
Dataset Summary
From Dissonance to Insights: Dissecting Disagreements in Rationale Construction for Case Outcome Classification
In legal NLP, Case Outcome Classification (COC) must not only be accurate but also trustworthy and explainable. Existing work in explainable COC has been limited to annotations by a single expert. However, it is well-known that lawyers may disagree in their assessment of case facts. We hence collect a novel dataset RaVE:… See the full description on the dataset page: https://huggingface.co/datasets/sxu/RaVE_emnlp23.AudioMomentRetrievalFromLongAudio_DCASE2026EvaluationData
What is this?
This repository contains data for DCASE Challenge 2026 Task 6.
Audio and text features using CLAP and a sliding window, following the same feature extraction protocol as the CASTELLA dataset.
submission template
File structure
clap
└──dcase2026_evaluation_audio_{vid}.npz
clap_text
└──qiddcase2026_evaluation_q{qid}.npz
Raw audio files
If participants require the raw audio, please contact the… See the full description on the dataset page: https://huggingface.co/datasets/lighthouse-emnlp2024/AudioMomentRetrievalFromLongAudio_DCASE2026EvaluationData.EMNLP2023wikipedia-emnlpuncertainty-vlm-qwen2p5-emnlp