embedding-benchmark/DS1000
DS-1000 is a code generation benchmark with a thousand data science problems spanning seven Python libraries, such as NumPy and Pandas. It employs multi-criteria evaluation metrics, including functional correctness and surface-form constraints, resulting in a high-quality dataset with only 1.8% incorrect solutions among accepted Codex-002 predictions. Usage import datasets # Download the dataset queries = datasets.load_dataset("embedding-benchmark/DS1000", "queries") documents =… See the full description on the dataset page: https://huggingface.co/datasets/embedding-benchmark/DS1000.
DS-1000 is a code generation benchmark with a thousand data science problems spanning seven Python libraries, such as NumPy and Pandas. It employs multi-criteria evaluation metrics, including functional correctness and surface-form constraints, resulting in a high-quality dataset with only 1.8% incorrect solutions among accepted Codex-002 predictions.
Usage
import datasets
# Download the dataset
queries = datasets.load_dataset("embedding-benchmark/DS1000", "queries")
documents = datasets.load_dataset("embedding-benchmark/DS1000", "corpus")
pair_labels = datasets.load_dataset("embedding-benchmark/DS1000", "default")