CoolFace
Datasetpublic

jannalu/mbpp-longcontext

MBPP Long-Context Dataset Overview MBPP Long-Context is a benchmark dataset that combines coding problems from the MBPP (Mostly Basic Python Problems) dataset with long-context distractors from BABILong. This dataset evaluates code generation performance under long-context conditions, testing whether models can maintain coding ability with stuffed context. Dataset Structure Data Fields Each sample contains: Original MBPP… See the full description on the dataset page: https://huggingface.co/datasets/jannalu/mbpp-longcontext.

sourceHugging Faceapache-2.0updated 11mo agoView on Hugging Face
0likes91downloads
Dataset Card

MBPP Long-Context Dataset

Overview

MBPP Long-Context is a benchmark dataset that combines coding problems from the MBPP (Mostly Basic Python Problems) dataset with long-context distractors from BABILong. This dataset evaluates code generation performance under long-context conditions, testing whether models can maintain coding ability with stuffed context.

Dataset Structure

Data Fields

Each sample contains:

Original MBPP Fields
  • —task_id (int): Unique task identifier
  • —text (str): Problem description
  • —code (str): Reference solution
  • —test_list (List[str]): Test cases (assertions)
  • —test_setup_code (str): Optional setup code
  • —challenge_test_list (List[str]): Additional test cases
Long-Context Fields
  • —context (str): Prepended distractor text from BABILong, ranging from 0k to 1M.
  • —context_id (str): BABILong source identifier (e.g., "babilong128kqa1sample42")
  • —context_length_tokens (int): Token count using Llama tokenizer
Metadata
  • —code_length_chars (int): Reference solution length for difficulty tracking

Data Splits

All configurations follow the original MBPP split structure:

  • —test: 500 samples (primary evaluation set)
  • —train: 374 samples
  • —validation: 90 samples
  • —prompt: 10 samples (few-shot examples)

Creating the dataset

To avoid confounding variables, this dataset uses stratified random assignment, where:

  1. 1.Sort MBPP tasks by code length
  2. 2.Get text from BABILong qa1-qa10 splits
  3. 3.Duplicate contexts to match task count (974 samples)
  4. 4.Shuffle contexts and assign to sorted tasks

Source Datasets

MBPP (Mostly Basic Python Problems)

BABILong