datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
DS-1000 DS-1000 in simplified format
🔥 Check the leaderboard from Eval-Arena on our project page.
See testing code and more information (also the original fill-in-the-middle/Insertion format) in the DS-1000 repo.
Reformatting credits: Yuhang Lai, Sida Wang
DS1000DS-1000 is a code generation benchmark with a thousand data science problems spanning seven Python libraries, such as NumPy and Pandas. It employs multi-criteria evaluation metrics, including functional correctness and surface-form constraints, resulting in a high-quality dataset with only 1.8% incorrect solutions among accepted Codex-002 predictions.
Usage
import datasets
# Download the dataset
queries = datasets.load_dataset("embedding-benchmark/DS1000", "queries")
documents =… See the full description on the dataset page: https://huggingface.co/datasets/embedding-benchmark/DS1000.ds1000DS-1000 dataset annotated with the ground-truth library documentation, to enable evaluations for retrieval and retrieval-augmented code generation.
Please refer to [code-rag-bench] for more details
ds1000-sft-dscoder_qwen2-7B_eval-ds1000basemodel-qwen2-7B-eval-ds1000ds1000_fail_codescore