datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
code_generation
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
🏠 Home Page •
💻 GitHub Repository •
🏆 Leaderboard •
LiveCodeBench is a "live" updating benchmark for holistically evaluating code related capabilities of LLMs.
Particularly, it evaluates LLMs across a range of capabilties including code generation, self-repair, test output prediction, and code execution.
This is the code generation scenario of LiveCodeBench. It is also… See the full description on the dataset page: https://huggingface.co/datasets/livecodebench/code_generation.livecodebench-code_generation_liteimpossible_livecodebenchLiveCodeBench-Proexecution-v2livecodebench_code_generation_lite_parquet
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
🏠 Home Page •
💻 GitHub Repository •
🏆 Leaderboard •
📄 Paper
Change Log
Since LiveCodeBench is a continuously updated benchmark, we provide different versions of the dataset. Particularly, we provide the following versions of the dataset:
release_v1: The initial release of the dataset with problems released between May 2023 and Mar 2024 containing 400… See the full description on the dataset page: https://huggingface.co/datasets/marianna13/livecodebench_code_generation_lite_parquet.livecodebench-code_generation_liteLiveCodeBench-CPP
LiveCodeBench-CPP: An Extension of LiveCodeBench for Contamination Free Evaluation in C++
Overview
LiveCodeBench-CPP includes 454 problems from the release_v6 of LiveCodeBench, covering the period from October 2024 to May 2025. These problems are sourced from AtCoder (287 problems) and LeetCode (167 problems).
AtCoder Problems: These require generated solutions to read inputs from standard input (stdin) and write outputs to standard output (stdout). For unit testing, the… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/LiveCodeBench-CPP.test_generation
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
🏠 Home Page •
💻 GitHub Repository •
🏆 Leaderboard •
LiveCodeBench is a "live" updating benchmark for holistically evaluating code related capabilities of LLMs.
Particularly, it evaluates LLMs across a range of capabilties including code generation, self-repair, test output prediction, and code execution.
This is the code generation scenario of LiveCodeBench. It is also… See the full description on the dataset page: https://huggingface.co/datasets/livecodebench/test_generation.livecodebench
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
Note: This is a clone of livecodebench/code_generation_lite updated to work with recent versions of the datasets library. The original repository uses a Python loading script which is no longer supported. This version provides the same data using the standard JSONL format for compatibility.
Dataset Description
LiveCodeBench is a "live" updating benchmark for holistically… See the full description on the dataset page: https://huggingface.co/datasets/bzantium/livecodebench.execution
Dataset Card for "livecodebench-execute"
LiveCodeBench-v5dataenvgym-livecodebench-solutionslivecodebench_code_generation_litelivecodebench-v6livecodebench_lite_filteredlivecodebench-code_generation_litelivecodebench-plus
LiveCodeBench-v6-Plus
A curated coding benchmark of 91 problems selected by hardness/discrimination
(lcb-v6-plus). It combines two sources, all in one clean schema:
64 evolved problems — mutated/evolved variants from LiveCodeBench-v6
(each carries its seed_problem).
27 original problems — un-evolved AtCoder problems taken directly from
livecodebench/code_generation_lite
release v6 (seed_problem is null).
About BenchEvolver
The evolved problems were produced by… See the full description on the dataset page: https://huggingface.co/datasets/BenchEvolver/livecodebench-plus.qwen3-8b-livecodebench-subset-v6-n128ko-livecodebench
Ko-LiveCodeBench
This dataset is the livecodebench/code_generation_lite dataset with the question_content field translated to Korean.
Dataset Versions
The dataset provides multiple configurations (subsets) corresponding to different release versions:
release_v1: Problems released between May 2023 and Mar 2024 (400 problems)
release_v2: Problems released between May 2023 and May 2024 (511 problems)
release_v3: Problems released between May 2023 and Jul 2024 (612 problems)… See the full description on the dataset page: https://huggingface.co/datasets/bzantium/ko-livecodebench.LiveCodeBench_V2_UTRL_evalTest set for evaluating LLM-based unit test generation capabilities, built upon LiveCodeBench-v2.
problem_statement: Description of the programming problem in LivecCodeBench-v2.
gt_test_cases: Ground-truth test cases to evaluate the correctness of the arbitrary code solutions.
sampled_code: 64 code solutions sampled from Qwen3-4B and Qwen3-8B.
Following evaluation scheme in Lee et al., 2026, Unit test generated by LLMs can be evaluated by the following metrics:
Best-of-N improvement:… See the full description on the dataset page: https://huggingface.co/datasets/dgjun32/LiveCodeBench_V2_UTRL_eval.LiveCodeBench-CodeGenerationqwen3-8b-livecodebench-v6-n128Ag-LiveCodeBench-XThis repository contains the multi-PL variant of LiveCodeBench, prepared in the Agnostics project.
Find out more about the dataset and the related artifacts on the project website.
The easiest way to benchmark a model on this dataset is with our scripts.
livecodebench-executelivecodebench
LiveCodeBench for Code-LLaVA
This dataset contains the LiveCodeBench code generation benchmark prepared for
Code-LLaVA evaluation.
Source
Paper: LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
Repository: https://github.com/LiveCodeBench/LiveCodeBench
Version: release_v6 (May 2023 - Apr 2025, 1055 problems)
Dataset Structure
Two configurations are available:
memwrap: Problems with <|memory_start|> /… See the full description on the dataset page: https://huggingface.co/datasets/tonychenxyz/livecodebench.livecodebench-merging-leaderboard
LiveCodeBench v6 Evaluation Leaderboard
Evaluation results for cross-capability merging of OLMo-3 and OLMo-3.1 RL-Zero models on 454 coding problems.
Evaluation
We followed the evaluation guidelines and prompts from OLMo 3. Best effort was made to ensure reported numbers are as accurate as possible.
Code: pmahdavi/modal-eval
Leaderboard
Model
pass@4
pass@1
Loop Rate
Qwen/Qwen3-4B-Thinking-2507
54.6%
45.4%
0.4%
pmahdavi/Olmo-3-7B-Think-Math-Code… See the full description on the dataset page: https://huggingface.co/datasets/pmahdavi/livecodebench-merging-leaderboard.livecodebench-execute-v2livecodebench_lite_contaminatedr1_livecodebench