datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
code_generation_liteLiveCodeBench is a temporaly updating benchmark for code generation. Please check the homepage: https://livecodebench.github.io/.code_generation
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
🏠 Home Page •
💻 GitHub Repository •
🏆 Leaderboard •
LiveCodeBench is a "live" updating benchmark for holistically evaluating code related capabilities of LLMs.
Particularly, it evaluates LLMs across a range of capabilties including code generation, self-repair, test output prediction, and code execution.
This is the code generation scenario of LiveCodeBench. It is also… See the full description on the dataset page: https://huggingface.co/datasets/livecodebench/code_generation.loom-benchmark-livecodebenchlivecodebench-code_generation_liteLiveCodeBench-Pro-Testcaseimpossible_livecodebenchLiveCodeBench-Proexecution-v2livecodebench_code_generation_lite_parquet
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
🏠 Home Page •
💻 GitHub Repository •
🏆 Leaderboard •
📄 Paper
Change Log
Since LiveCodeBench is a continuously updated benchmark, we provide different versions of the dataset. Particularly, we provide the following versions of the dataset:
release_v1: The initial release of the dataset with problems released between May 2023 and Mar 2024 containing 400… See the full description on the dataset page: https://huggingface.co/datasets/marianna13/livecodebench_code_generation_lite_parquet.livecodebench-code_generation_liteLiveCodeBench-CPP
LiveCodeBench-CPP: An Extension of LiveCodeBench for Contamination Free Evaluation in C++
Overview
LiveCodeBench-CPP includes 454 problems from the release_v6 of LiveCodeBench, covering the period from October 2024 to May 2025. These problems are sourced from AtCoder (287 problems) and LeetCode (167 problems).
AtCoder Problems: These require generated solutions to read inputs from standard input (stdin) and write outputs to standard output (stdout). For unit testing, the… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/LiveCodeBench-CPP.test_generation
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
🏠 Home Page •
💻 GitHub Repository •
🏆 Leaderboard •
LiveCodeBench is a "live" updating benchmark for holistically evaluating code related capabilities of LLMs.
Particularly, it evaluates LLMs across a range of capabilties including code generation, self-repair, test output prediction, and code execution.
This is the code generation scenario of LiveCodeBench. It is also… See the full description on the dataset page: https://huggingface.co/datasets/livecodebench/test_generation.livecodebench
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
Note: This is a clone of livecodebench/code_generation_lite updated to work with recent versions of the datasets library. The original repository uses a Python loading script which is no longer supported. This version provides the same data using the standard JSONL format for compatibility.
Dataset Description
LiveCodeBench is a "live" updating benchmark for holistically… See the full description on the dataset page: https://huggingface.co/datasets/bzantium/livecodebench.execution
Dataset Card for "livecodebench-execute"
LiveCodeBench-v5dataenvgym-livecodebench-solutionslivecodebench_code_generation_litelivecodebench-v6submissions
LiveCodeBench Submissions
To submit your models to LiveCodeBench, you can now use this huggingface repository to directly upload model generations. To submit the model generations, simply drag and drop your model generations folder here and create a pull request.
livecodebench_lite_filteredlivecodebench-code_generation_litelivecodebench-plus
LiveCodeBench-v6-Plus
A curated coding benchmark of 91 problems selected by hardness/discrimination
(lcb-v6-plus). It combines two sources, all in one clean schema:
64 evolved problems — mutated/evolved variants from LiveCodeBench-v6
(each carries its seed_problem).
27 original problems — un-evolved AtCoder problems taken directly from
livecodebench/code_generation_lite
release v6 (seed_problem is null).
About BenchEvolver
The evolved problems were produced by… See the full description on the dataset page: https://huggingface.co/datasets/BenchEvolver/livecodebench-plus.qwen3-8b-livecodebench-subset-v6-n128ko-livecodebench
Ko-LiveCodeBench
This dataset is the livecodebench/code_generation_lite dataset with the question_content field translated to Korean.
Dataset Versions
The dataset provides multiple configurations (subsets) corresponding to different release versions:
release_v1: Problems released between May 2023 and Mar 2024 (400 problems)
release_v2: Problems released between May 2023 and May 2024 (511 problems)
release_v3: Problems released between May 2023 and Jul 2024 (612 problems)… See the full description on the dataset page: https://huggingface.co/datasets/bzantium/ko-livecodebench.LiveCodeBench_V2_UTRL_evalTest set for evaluating LLM-based unit test generation capabilities, built upon LiveCodeBench-v2.
problem_statement: Description of the programming problem in LivecCodeBench-v2.
gt_test_cases: Ground-truth test cases to evaluate the correctness of the arbitrary code solutions.
sampled_code: 64 code solutions sampled from Qwen3-4B and Qwen3-8B.
Following evaluation scheme in Lee et al., 2026, Unit test generated by LLMs can be evaluated by the following metrics:
Best-of-N improvement:… See the full description on the dataset page: https://huggingface.co/datasets/dgjun32/LiveCodeBench_V2_UTRL_eval.livecodebench-venv-portable-bundle
LiveCodeBench Portable Venv Bundle
This directory contains a portable transfer bundle for the LiveCodeBench Python environment.
Files
LiveCodeBench-venv-portable-20260407.tar.zst
Portable environment archive.
LiveCodeBench-venv-portable-20260407.tar.zst.sha256
SHA256 checksum file for archive verification.
restore_livecodebench_venv.sh
Restore the environment on the target server and rewrite internal absolute paths.
send_livecodebench_venv_to_server.sh
Copy the bundle to… See the full description on the dataset page: https://huggingface.co/datasets/tokenintelligence/livecodebench-venv-portable-bundle.LiveCodeBench-CodeGenerationqwen3-8b-livecodebench-v6-n128LiveCodeBench-ReasonFluxWe use Stdio input/output format here. For example, for the task to calculate the sum of a list, the input and output are in the following format:
input = "5\n1 2 3 4 5\n"
output = "15"
CodeContests and CodeForces are using this format, however, MBPP and part of LiveCodeBench are using functional input/output format, such like
assert sum_function([1, 2, 3, 4, 5]) == 15
In this project, we have converted the the functional format to the Stdio format to achieve consistency.
Paper | Code… See the full description on the dataset page: https://huggingface.co/datasets/Gen-Verse/LiveCodeBench-ReasonFlux.Ag-LiveCodeBench-XThis repository contains the multi-PL variant of LiveCodeBench, prepared in the Agnostics project.
Find out more about the dataset and the related artifacts on the project website.
The easiest way to benchmark a model on this dataset is with our scripts.