datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
jbdIVR-pilot-benchmark
IntentSpec Benchmark — Data Supplement
This archive contains the benchmark data used to compute Intent Violation
Rate (IVR) in the paper: 49 tasks, each derived from a HumanEval problem and
extended with an ambiguous/gold prompt pair and a decomposed set of
executable constraints.
Files
spec_pairs.jsonl
The benchmark itself — one JSON object per line, one line per task. This is
the file consumed directly by the evaluation pipeline… See the full description on the dataset page: https://huggingface.co/datasets/anote-ai/IVR-pilot-benchmark.ivrit-manifestsivrit-manifests-textsivrit-recordings