datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
mbpp
Dataset Card for Mostly Basic Python Problems (mbpp)
Dataset Summary
The benchmark consists of around 1,000 crowd-sourced Python programming problems, designed to be solvable by entry level programmers, covering programming fundamentals, standard library functionality, and so on. Each problem consists of a task description, code solution and 3 automated test cases. As described in the paper, a subset of the data has been hand-verified by us.
Released here as part of… See the full description on the dataset page: https://huggingface.co/datasets/google-research-datasets/mbpp.mbppplusmbppThe MBPP (Mostly Basic Python Problems) dataset consists of around 1,000 crowd-sourced Python
programming problems, designed to be solvable by entry level programmers, covering programming
fundamentals, standard library functionality, and so on. Each problem consists of a task
description, code solution and 3 automated test cases.multilingual_mbppMBPP translated to 15 programming languages using o4-mini-medium.
source_language = "python"
target_languages = [
"cpp",
"c",
"javascript",
"java",
"php",
"csharp",
"typescript",
"bash",
"swift",
"go",
"rust",
"ruby",
"r",
"matlab",
"scala",
"haskell"
]
effort = "medium"
dataset_name = "google-research-datasets/mbpp"
model = "o4-mini"
mbpp-synthetic-v3mbpp-hardcodembpp-multilingualmbpp_extended
Dataset Card for "mbpp_extended"
More Information needed
loom-benchmark-mbpphumaneval-mbpp-codegen-qa
Dataset Card for "humaneval-mbpp-codegen-qa"
This dataset contains prompt-reply (question-answer) pairs where the prompt is to create a Python function which satisfies the functionality described in a specified docstring. The responses are then the generated functions.
mbpp
Dataset Card for Mostly Basic Python Problems (mbpp)
Dataset Summary
The benchmark consists of around 1,000 crowd-sourced Python programming problems, designed to be solvable by entry level programmers, covering programming fundamentals, standard library functionality, and so on. Each problem consists of a task description, code solution and 3 automated test cases. As described in the paper, a subset of the data has been hand-verified by us.
Released here as part of… See the full description on the dataset page: https://huggingface.co/datasets/nlile/mbpp.PPE-MBPP-Plus-Best-of-K
Overview
This contains the MBPP-Plus correctness preference evaluation set for Preference Proxy Evaluations.
The prompts are sampled from MBPP-Plus.
This dataset is meant for benchmarking and evaluation, not for training.
Paper
Code
License
User prompts are licensed under Apache-2.0, and model outputs are governed by the terms of use set by the respective model providers.
Citation
@misc{frick2024evaluaterewardmodelsrlhf,
title={How to Evaluate… See the full description on the dataset page: https://huggingface.co/datasets/lmarena-ai/PPE-MBPP-Plus-Best-of-K.MBPPThe MBPP dataset consists of around 1,000 crowd-sourced Python programming problems, designed to be solvable by entry level programmers, covering programming fundamentals, standard library functionality, and so on. Each problem consists of a task description, code solution and 3 automated test cases. As described in the paper, a subset of the data has been hand-verified by the dataset authors to ensure quality.
Usage
import datasets
# Download the dataset
queries =… See the full description on the dataset page: https://huggingface.co/datasets/embedding-benchmark/MBPP.pybytecode-mbpp-3.12
Data card — MBPP held-out decompilation benchmark
383 MBPP reference solutions compiled to Python 3.12 bytecode and paired with their source, with
the original task_id restored on every row. Built 2026-08-04 by tools/build_mbpp_ood.py.
Redistributable under CC-BY-4.0, provided NOTICES.md ships alongside.
The set
Rows
383 (400 candidates − 17 contaminated)
Source
google-research-datasets/mbpp, config full
Licence
CC-BY-4.0
task_id restored
383 /… See the full description on the dataset page: https://huggingface.co/datasets/BlazingCustoms/pybytecode-mbpp-3.12.bc-mbppThe MBPP dataset in BabelCode format.mbpp
Information
This is a reformatted version of the HumanEval dataset
OpenCodeInstruct-MBPP-TextHumanEval_mbpp_format
Dataset Card for "HumanEval_mbpp_format"
More Information needed
mbpp_short
Dataset Card for "mbpp_short"
More Information needed
impossible_mbpp_natural_diversembpp-ja
mbpp-ja
This repository provides a mbpp dataset translated from English into Japanese by LLM-jp, a collaborative project launched in Japan.
For English to Japanese translation, DeepL was used.
The links of the original mbpp dataset are here(HuggingFace) or here(GitHub).
Send Questions to
llm-jp(at)nii.ac.jp
Model Card Authors
The names are listed in alphabetical order.
Namgi Han, Masatoshi Otake, Shintaro Ozaki, Yusuke Miyao.
mbppThis is the MBPP dataset. Downloaded from here
and constructed as follows:
import datasets
ds = datasets.load_dataset("json", data_files="mbpp.jsonl", split="train")
test = ds.filter(lambda item: item['task_id'] >= 11 and item['task_id'] <= 510)
few_shot = ds.filter(lambda item: item['task_id'] >= 1 and item['task_id'] <= 10)
validation = ds.filter(lambda item: item['task_id'] >= 511 and item['task_id'] <= 600)
train = ds.filter(lambda item: item['task_id'] >= 601 and item['task_id'] <= 974)… See the full description on the dataset page: https://huggingface.co/datasets/arjunguha/mbpp.mbpp-agnostic-translationA dataset developed during the Agnostics project.
See the Agnostics framework repository
for code which dynamically generates the Ag-MBPP-X datasets from this underlying dataset.
Also see the HF collection grouping datasets from the project.
dfg_augmented_mbpp
Dataset Card for "dfg_augmented_mbpp"
More Information needed
mbpp-tr
MBPP-TR
MBPP (Mostly Basic Python Problems)
veri setinin Türkçe çevirisi. Orijinal veri setindeki gibi iki config içerir: sanitized ve full.
Her satırda görev açıklamasının İngilizce aslı ve Türkçe çevirisi birlikte yer alır; kod ve testler
orijinal veri setinden değiştirilmeden aktarılmıştır.
A Turkish translation of MBPP with both the sanitized and full configs. Each row contains the
original English task description and its Turkish translation; code and tests are copied… See the full description on the dataset page: https://huggingface.co/datasets/firatmio/mbpp-tr.mbpp_with_promptmbpp-code-rl
MBPP for code RL (deduplicated against MBPP+)
MBPP prepared for RLVR training in verl,
with two independent hold-outs so both MBPP+ and MBPP's own canonical test
split stay reportable after training on this data.
split
rows
contents
train
320
MBPP canonical train + validation + prompt, minus everything in MBPP+
test
378
exactly the problems in evalplus/mbppplus
heldout_mbpp_test
276
MBPP's canonical test split (task_id 11-510) that is not in MBPP+… See the full description on the dataset page: https://huggingface.co/datasets/RL-Forgetting-Experiments-3/mbpp-code-rl.mbpp-proEvaluation dataset for umanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generation Task (arxiv.org/abs/2412.21199).
mbpp-longcontext
MBPP Long-Context Dataset
Overview
MBPP Long-Context is a benchmark dataset that combines coding problems from the MBPP (Mostly Basic Python Problems) dataset with long-context distractors from BABILong. This dataset evaluates code generation performance under long-context conditions, testing whether models can maintain coding ability with stuffed context.
Dataset Structure
Data Fields
Each sample contains:
Original MBPP Fields… See the full description on the dataset page: https://huggingface.co/datasets/jannalu/mbpp-longcontext.mbpp
Dataset Card for Mostly Basic Python Problems (mbpp)
Dataset Summary
The benchmark consists of around 1,000 crowd-sourced Python programming problems, designed to be solvable by entry level programmers, covering programming fundamentals, standard library functionality, and so on. Each problem consists of a task description, code solution and 3 automated test cases. As described in the paper, a subset of the data has been hand-verified by us.
Released here as part of… See the full description on the dataset page: https://huggingface.co/datasets/RLAIF/mbpp.