soundness
SoundnessBench
SoundnessBench
Overview
This repository contains the datasets and pretrained model checkpoints used in SoundnessBench, a benchmark designed to thoroughly evaluate the soundness of neural network (NN) verifiers.
SoundnessBench aims to support developers in evaluating and improving NN verifiers.
For more detailed usage instruction, see our github repository for more information.
Dataset Information
The downloaded benchmark should contain a total of 26 models… See the full description on the dataset page: https://huggingface.co/datasets/SoundnessBench/SoundnessBench.SoundnessBench
SoundnessBench
SoundnessBench is a benchmark of 1,099 machine learning research proposals reconstructed from ICLR submissions and labeled with reviewer soundness sub-scores.
It is intended to evaluate whether LLMs can judge proposal-stage methodological soundness before expensive experimentation.
Data
The dataset file is:
soundnessbench.jsonl
Expected local layout:
data/soundnessbench.jsonl
Evaluation
Reference code and prompts live here:… See the full description on the dataset page: https://huggingface.co/datasets/hosytuyen/SoundnessBench.soundness_data_integrity.jsonsoundness_logic_validation.jsonsoundness_audio_events.json