datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
DS-1000 DS-1000 in simplified format
🔥 Check the leaderboard from Eval-Arena on our project page.
See testing code and more information (also the original fill-in-the-middle/Insertion format) in the DS-1000 repo.
Reformatting credits: Yuhang Lai, Sida Wang
DS1000DS-1000 is a code generation benchmark with a thousand data science problems spanning seven Python libraries, such as NumPy and Pandas. It employs multi-criteria evaluation metrics, including functional correctness and surface-form constraints, resulting in a high-quality dataset with only 1.8% incorrect solutions among accepted Codex-002 predictions.
Usage
import datasets
# Download the dataset
queries = datasets.load_dataset("embedding-benchmark/DS1000", "queries")
documents =… See the full description on the dataset page: https://huggingface.co/datasets/embedding-benchmark/DS1000.ds1000-test-rollouts
DS-1000 test-suite rollouts
Model-generated test suites for all 1000 DS-1000
problems, for the joint coding-and-testing construction (independent code and test experts;
see the mlcb-ocaml campaign).
The matching code rollouts are the domain=ds1000 cells of
samuki-hf/thinking-rollouts
(join on instance_id = DS-1000 test-split row index).
100 rollouts per problem per model: Qwen3-4B / 8B / 14B, non-thinking
(enable_thinking=False), temperature 0.6, top_p 0.8, top_k 20, min_p 0.0… See the full description on the dataset page: https://huggingface.co/datasets/vxef/ds1000-test-rollouts.rteb-DS1000Retrieval
DS1000Retrieval — RTEB open subset, unified schema
A normalised copy of the dataset behind the mteb task DS1000Retrieval, one of the 17 open tasks in the
RTEB(beta) retrieval benchmark. Same queries, documents and relevance
judgements as the benchmark evaluates — reshaped into one strict schema shared by all 17.
Source
embedding-benchmark/DS1000 @ 25cd4dc8172e (the revision pinned in mteb)
Domain · languages
code · eng
Queries / documents / qrels
1,998 / 1,998 /… See the full description on the dataset page: https://huggingface.co/datasets/Hyukkyu/rteb-DS1000Retrieval.ds1000_pnyx
PNYX - DS-1000
This is a splitted and tested version of DS-1000, based on the reformatted version claudios/ds1000 (extracted metadata as columns). This version is designed to be compatible with the hf_evaluate code_eval package. Also, the code was modified to work with newer versions of the used python packages (numpy, scipy, etc.).
This dataset includes all the original fields and the following ones:
user_chat_prompt: A chat-style prompt for the problem, adapted from the prompt… See the full description on the dataset page: https://huggingface.co/datasets/PNYX/ds1000_pnyx.ds1000DS-1000 dataset annotated with the ground-truth library documentation, to enable evaluations for retrieval and retrieval-augmented code generation.
Please refer to [code-rag-bench] for more details
ds1000-sDS1000Retrieval
DS1000Retrieval
An MTEB dataset
Massive Text Embedding Benchmark
A code retrieval task based on 1,000 data science programming problems from DS-1000. Each query is a natural language description of a data science task (e.g., 'Create a scatter plot of column A vs column B with matplotlib'), and the corpus contains Python code implementations using libraries like pandas, numpy, matplotlib, scikit-learn, and scipy. The task is to retrieve the correct code snippet that solves the… See the full description on the dataset page: https://huggingface.co/datasets/mteb/DS1000Retrieval.ds1000This is a reupload of DS-1000. The metadata dictionary has been extracted into columns and the categorical variables are now ClassLabel types, and the dataset is natively a parquet. The features are as follows:
Column
Type
problem_id
Value(dtype='int64', id=None)
prompt
Value(dtype='string', id=None)
reference_code
Value(dtype='string', id=None)
code_context
Value(dtype='string', id=None)
library_problem_id
Value(dtype='int64', id=None)
library… See the full description on the dataset page: https://huggingface.co/datasets/claudios/ds1000.ds1000_qwen7b_att_iter0_ppo_att20_sol10dream_coder_trajectory_ds1000ds1000-instruction-output
Dataset Card for "ds1000-instruction-output"
More Information needed
ds1000_humaneval_qwen7b_att_iter0ds1000_humaneval_qwen7b_att_iter0_dpo_3000ds1000-val-100ds1000_qwen32b_att_iter0_ppo_att20_sol10ds1000_qwen7b_sol_iter0_ppo_att20_sol10ds1000_qwen7b_sol_iter0_ppo_att20_sol10_dpo_5000ds1000_all_inputds1000_qwen7b_att_iter0_ppo_att2_sol2pro-r8-ds1000-mirrords1000_humaneval_qwen7b_att_iter0_dpo_5000ft-dscoder_qwen2-7B_eval-ds1000basemodel-qwen2-7B-eval-ds1000ds1000_fail_codescore