datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
cornstack-python-v1
CoRNStack Python Dataset
The CoRNStack Dataset, accepted to ICLR 2025, is a large-scale high quality training dataset specifically for code retrieval across multiple
programming languages. This dataset comprises of <query, positive, negative> triplets used to train nomic-embed-code,
CodeRankEmbed, and CodeRankLLM.
CoRNStack Dataset Curation
Starting with the deduplicated Stackv2, we create text-code pairs from function docstrings and respective code. We filtered out… See the full description on the dataset page: https://huggingface.co/datasets/nomic-ai/cornstack-python-v1.python-codes-25k
License
MIT
This is a Cleaned Python Dataset Covering 25,000 Instructional Tasks
Overview
The dataset has 4 key features (fields): instruction, input, output, and text.It's a rich source for Python codes, tasks, and extends into behavioral aspects.
Dataset Statistics
Total Entries: 24,813
Unique Instructions: 24,580
Unique Inputs: 3,666
Unique Outputs: 24,581
Unique Texts: 24,813
Average Tokens per example: 508
Features… See the full description on the dataset page: https://huggingface.co/datasets/flytech/python-codes-25k.python-toolcallsLogs from run_python_code tool used for benchmarking.
rl-pythonTested-143k-Python-AlpacaContributors: Nicolas Mejia Petit
Vezora's CodeTester Dataset
Introduction
Today, on March 6, 2024, we are excited to release our internal Python dataset with 143,327 examples of code. These examples have been meticulously tested and verified as working. Our dataset was created using a script we developed.
Dataset Creation
Our script operates by extracting Python code from the output section of Alpaca-formatted datasets. It tests each extracted piece of code… See the full description on the dataset page: https://huggingface.co/datasets/Vezora/Tested-143k-Python-Alpaca.github-repos-pythonThe Github repository retrieval source for [code-rag-bench], containing all Python files from the entire GitHub dump (in github-repos)
Tested-22k-Python-AlpacaContributors: Nicolas Mejia Petit
Vezora's CodeTester Dataset
Introduction
Today, on November 2, 2023, we are excited to release our internal Python dataset with 22,600 examples of code. These examples have been meticulously tested and verified as working. Our dataset was created using a script we developed.
Dataset Creation
Our script operates by extracting Python code from the output section of Alpaca-formatted datasets. It tests each extracted piece of… See the full description on the dataset page: https://huggingface.co/datasets/Vezora/Tested-22k-Python-Alpaca.starcoder-python-instruct
StarCoder-Python-Qwen-Instruct
Dataset Description
This dataset contains Python code samples paired with synthetically generated natural language instructions. It is designed for supervised fine-tuning of language models for code generation tasks. The dataset is derived from the Python subset of the bigcode/starcoderdata corpus, and the instructional text for each code sample was generated using the Qwen/Qwen3-Coder-30B-A3B-Instruct-FP8 model.
Creation… See the full description on the dataset page: https://huggingface.co/datasets/OLMo-Coding/starcoder-python-instruct.Python-Code-23k-ShareGPTThis dataset is in Vicuna/ShareGPT format. There are 23000+ set of conversations. Each set having 2 conversations.
Along with the Python code detailed explanation is provided.
This dataset was generated using GPT-3.5, GPT-4 etc.
python_functions_reasoningThis is the Python (functions) coding reasoning dataset used to train
Notbad v1.0 Mistral 24B reasoning model.
The reasoning data were sampled from an RL-based self-improved
Mistral-Small-24B-Instruct-2501 model.
The Python functions and instructions were sourced from OpenCoder Dataset Stage1
and from open source projects on Github.
You can try Notbad v1.0 Mistral 24B on chat.labml.ai.
Python-Code-LargePython-Code-Large
Python-Code-Large is a large-scale corpus of Python source code comprising more than 2 million rows of Python code. The dataset is designed to support research in large language model (LLM) pretraining, code intelligence, software engineering automation, and program analysis for the Python ecosystem.
By providing a high-volume, language-specific corpus, Python-Code-Large enables systematic experimentation in Python-focused model training, domain adaptation, and downstream… See the full description on the dataset page: https://huggingface.co/datasets/Lovett01/Python-Code-Large.Python-Code-LargePython-Code-Large
Python-Code-Large is a large-scale corpus of Python source code comprising more than 2 million rows of Python code. The dataset is designed to support research in large language model (LLM) pretraining, code intelligence, software engineering automation, and program analysis for the Python ecosystem.
By providing a high-volume, language-specific corpus, Python-Code-Large enables systematic experimentation in Python-focused model training, domain adaptation, and downstream… See the full description on the dataset page: https://huggingface.co/datasets/ajibawa-2023/Python-Code-Large.Competitive-Programming-python-blend
Dataset Card for Competitive-Programming-python-blend
Summary
Competitive-Programming-python-blend is a mixed supervised fine-tuning dataset centered on competitive programming, code reasoning, and instruction-style problem solving. The blend is Python-first, but it also keeps a small amount of C++, agentless SWE, and reasoning-oriented chat supervision to broaden training coverage.
The current release is published as a single HF-friendly JSONL file, clean.jsonl.… See the full description on the dataset page: https://huggingface.co/datasets/Jackrong/Competitive-Programming-python-blend.Trajectory-Stitching-Test-7M
Dataset Creation & Methodology
Building this dataset required a highly optimized pipeline running on a dual-H100 NVL GPU cluster. The stitching process operates autonomously without relying on external LLM calls, using a specialized two-pass algorithm.
1. High-Information Keyword Extraction
Instead of relying on simple word counts, the pipeline dynamically builds a dataset-specific stopword list by analyzing Document Frequency (DF) to banish words appearing in more than… See the full description on the dataset page: https://huggingface.co/datasets/pythonformer/Trajectory-Stitching-Test-7M.python-code-instructions-85k
Python Code Instructions - 85K
Instruction-tuning dataset of Python functions paired with short natural-language instructions derived from repository docstrings.
What changed in this release
This release keeps the original public rows and format, but makes the dataset easier to use responsibly:
exact duplicate rows were removed again using normalized instruction + output hashing
deterministic train, validation, and test splits were added
the dataset card now documents… See the full description on the dataset page: https://huggingface.co/datasets/NickIBrody/python-code-instructions-85k.code-text-python
Dataset is imported from CodeXGLUE and pre-processed using their script.
Where to find in Semeru:
The dataset can be found at /nfs/semeru/semeru_datasets/code_xglue/code-to-text/python in Semeru
CodeXGLUE -- Code-To-Text
Task Definition
The task is to generate natural language comments for a code, and evaluted by smoothed bleu-4 score.
Dataset
The dataset we use comes from CodeSearchNet and we filter the dataset as the following:
Remove… See the full description on the dataset page: https://huggingface.co/datasets/semeru/code-text-python.floorplans-cityscapes
Dataset Summary
This is a curated collection of floorplan images sourced from across the internet. It is intended for research in architectural AI, layout generation, and urban scene understanding.
Data format: Image files with associated integer labels.
Sources: Publicly available images from various web sources (This dataset is one unified collections).
Purpose: Educational and research use.
Dataset Structure
The dataset follows the standard Hugging Face Image… See the full description on the dataset page: https://huggingface.co/datasets/wheres-my-python/floorplans-cityscapes.thai-culturax-clean-dataset
Thai CulturaX Clean dataset
The data is sourced from the Thai subset of CulturaX dataset, which itself is sourced from mC4 and four OSCAR corpora.
It has about 8,748,575,684 words (without whitespace) and 16,768,585 lines (97 GB).
It was filtered content promoting gambling, adult content, and narcotics.
GitHub for clean: https://github.com/wannaphong/thai-filter-website
Considerations for Using the Data
This dataset is the cleaned version of the CulturaX datasets… See the full description on the dataset page: https://huggingface.co/datasets/pythainlp/thai-culturax-clean-dataset.LLMcoder-GitHub-Python-Mix-Direct
Dataset Card for LLMcoder-GitHub-Python-Mix-Direct
Python target autocomplete suggestions in the format of conversations for OpenAI's fine-tuning.
Dataset Details
Dataset Description
Curated by: [More Information Needed]
Funded by [optional]: [More Information Needed]
Shared by [optional]: [More Information Needed]
Language(s) (NLP): [More Information Needed]
License: [More Information Needed]
Dataset Sources [optional]
The data has been… See the full description on the dataset page: https://huggingface.co/datasets/23ws-LLMcoder/LLMcoder-GitHub-Python-Mix-Direct.python_codeStable-Code-Python-SFT
Stable Code Python SFT
The Stable Code Python SFT dataset is a high-quality synthetic dataset derived from the
stabilityai/stable-code-instruct-3b model for the purpose of supervised fine-tuning (SFT). Please refer to the
Versioning section for dataset versions.
Note: If you would like to contribute to this repository,
please read the CONTRIBUTING first.
TableofContents
Features
File Structure
Metadata
Usage
Versioning
License
TeamContact
Reference
Citation… See the full description on the dataset page: https://huggingface.co/datasets/bunyaminergen/Stable-Code-Python-SFT.leetcode-python-dataset
leetcode-python-dataset
Code for building and publishing the justindal/leetcode-python-dataset dataset on Hugging Face.
Merges two open-source LeetCode datasets into a unified schema with consistent formatting, field normalisation, and solution validation.
Dataset
Split
Rows
Source
train
2856
newfacade + greengerong
valid
310
slug-group split from train
test
228
newfacade only
Schema
default config (training)
Each row is a… See the full description on the dataset page: https://huggingface.co/datasets/justindal/leetcode-python-dataset.python_enhancement_proposals
Python Enhancement Proposals
Description
Python Enhancement Proposals, or PEPs, are design documents that generally provide a technical specification and rationale for new features of the Python programming language.
There are been 661 PEPs published.
The majority of PEPs are published in the Public Domain, but 5 were published under the “Open Publication License” and omitted from this dataset.
PEPs are long, highly-polished, and technical in nature and often include… See the full description on the dataset page: https://huggingface.co/datasets/common-pile/python_enhancement_proposals.python-unit-test-training-pool
Python unit test training pool
A pool of public data for training a model to write tests for Python code. It is a
straight collection of open datasets, not a new corpus: every row comes from one of the
sources below, at the revision named, and the only rows removed are the ones an overlap
filter flagged against held-out material this pool is kept separate from.
Every row of the normalised layer pairs a program with tests for it. That is the point of
the pool, and it is why the… See the full description on the dataset page: https://huggingface.co/datasets/Emulated-Inc/python-unit-test-training-pool.SFT_Dataset
Pythagoras SFT Dataset
Project Page | GitHub | Paper
Data
Our training dataset consists of approximately 841K problems paired with Lean formal statements, formal proofs, and reasoning chains. We release a partial subset, which consists of 126K instances:
30K easy instances
49K medium instances
47K hard instances
Complete data will be released soon.
The complete explanation of the synthetic data generation pipeline can be found in Pythagoras-Prover: Advancing… See the full description on the dataset page: https://huggingface.co/datasets/Pythagoras-LM/SFT_Dataset.python_code_docstring_ast_corpus
Overview
This dataset contains 34,000+ rows of code-docstring-ast data along with additional metadata. Data was gathered from various Python libraries and frameworks and their
publicly available GitHub repos. This dataset was created for the purpose of training the CodeT5+ transformer on AST-enhanced code-to-doc tasks.
Sources
The dataset was gathered from various GitHub repos sampled from this repo by Vinta.
The 26 repos are:
matplotlib
pytorch
cryptography
django… See the full description on the dataset page: https://huggingface.co/datasets/Mir-2002/python_code_docstring_ast_corpus.codeagent-pythonPython_Refactor_Dataset
🧩 Python Refactor Dataset (45k)
Behavior-Preserving Refactoring Examples for Instruction-Tuning Code Models
This dataset contains 45,000 synthetic Python code refactoring examples designed for
instruction-tuning models such as IBM Granite 4.0 (micro/h-tiny) and Meta CodeLlama-7B-Python.
Each example demonstrates a behavior-preserving refactor — improving code readability,
maintainability, and style (PEP8, type hints, context managers, modularization, etc.)
without… See the full description on the dataset page: https://huggingface.co/datasets/KavinduHansaka/Python_Refactor_Dataset.python_documentation_codeDataset created using https://github.com/jeffmeloy/py2dataset using the Python code from the following:
https://github.com/ansible/ansible
https://github.com/apache/airflow
https://github.com/arogozhnikov/einops
https://github.com/arviz-devs/arviz
https://github.com/astropy/astropy
https://github.com/biopython/biopython
https://github.com/bjodah/chempy
https://github.com/bokeh/bokehhttps://github.com/CalebBell/thermo
https://github.com/camDavidsonPilon/lifelines
https://github.com/coin-or/pulp… See the full description on the dataset page: https://huggingface.co/datasets/jeffmeloy/python_documentation_code.29K_Python_Docstring_Pairs
29K High-Quality Python Docstring Pairs
Author: Michael Hernandez (XxCotHGxX)License: CC BY 4.0Cleaned from: XxCotHGxX/242K_Python_Docstring_Pairs
Overview
A curated, high-quality subset of Python function–docstring pairs for use in code documentation generation, docstring completion, and code understanding tasks.
The original 242K dataset was scraped from open-source Python repositories but contained a significant proportion of functions without docstrings (84% of… See the full description on the dataset page: https://huggingface.co/datasets/XxCotHGxX/29K_Python_Docstring_Pairs.