anote-ai/IVR-pilot-benchmark
IntentSpec Benchmark — Data Supplement This archive contains the benchmark data used to compute Intent Violation Rate (IVR) in the paper: 49 tasks, each derived from a HumanEval problem and extended with an ambiguous/gold prompt pair and a decomposed set of executable constraints. Files spec_pairs.jsonl The benchmark itself — one JSON object per line, one line per task. This is the file consumed directly by the evaluation pipeline… See the full description on the dataset page: https://huggingface.co/datasets/anote-ai/IVR-pilot-benchmark.
Conversations for this repository live on Hugging Face.
CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.
Open discussions on Hugging Face