datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
python_code_instructions_18k_alpaca
Dataset Card for python_code_instructions_18k_alpaca
The dataset contains problem descriptions and code in python language.
This dataset is taken from sahil2801/code_instructions_120k, which adds a prompt column in alpaca style. Refer to the source here.
python-codes-25k
License
MIT
This is a Cleaned Python Dataset Covering 25,000 Instructional Tasks
Overview
The dataset has 4 key features (fields): instruction, input, output, and text.It's a rich source for Python codes, tasks, and extends into behavioral aspects.
Dataset Statistics
Total Entries: 24,813
Unique Instructions: 24,580
Unique Inputs: 3,666
Unique Outputs: 24,581
Unique Texts: 24,813
Average Tokens per example: 508
Features… See the full description on the dataset page: https://huggingface.co/datasets/flytech/python-codes-25k.code-search-net-python
Dataset Card for "code-search-net-python"
Dataset Description
Homepage: None
Repository: https://huggingface.co/datasets/Nan-Do/code-search-net-python
Paper: None
Leaderboard: None
Point of Contact: @Nan-Do
Dataset Summary
This dataset is the Python portion of the CodeSarchNet annotated with a summary column.The code-search-net dataset includes open source functions that include comments found at GitHub.The summary is a short description of what the… See the full description on the dataset page: https://huggingface.co/datasets/Nan-Do/code-search-net-python.python-code-dataset-500k
Attention: This dataset is a summary and reformat pulled from github code.
You should make your own assumptions based on this.
In fact, there is another dataset I formed through parsing that addresses several points:
out of 500k python related items, most of them are python-ish, not pythonic
the majority of the items here contain excessive licensing inclusion of original code
the items here are sometimes not even python but have references
There's a whole lot of gpl summaries… See the full description on the dataset page: https://huggingface.co/datasets/jtatman/python-code-dataset-500k.CodeExercise-Python-27k
Dataset Card for CodeFuse-CodeExercise-Python-27k
[中文] [English]
Dataset Description
This dataset consists of 27K Python programming exercises (in English), covering hundreds of Python-related topics including basic syntax and data structures, algorithm applications, database queries, machine learning, and more.
Please note that this dataset was generated with the help of a teacher model and Camel, and has not undergone strict validation. There may be errors or… See the full description on the dataset page: https://huggingface.co/datasets/codefuse-ai/CodeExercise-Python-27k.instructional_code-search-net-python
Dataset Card for "instructional_code-search-net-python"
Dataset Summary
This is an instructional dataset for Python.
The dataset contains two different kind of tasks:
Given a piece of code generate a description of what it does.
Given a description generate a piece of code that fulfils the description.
Languages
The dataset is in English.
Data Splits
There are no splits.
Dataset Creation
May of 2023
Curation Rationale
This… See the full description on the dataset page: https://huggingface.co/datasets/Nan-Do/instructional_code-search-net-python.Python-Code-LargePython-Code-Large
Python-Code-Large is a large-scale corpus of Python source code comprising more than 2 million rows of Python code. The dataset is designed to support research in large language model (LLM) pretraining, code intelligence, software engineering automation, and program analysis for the Python ecosystem.
By providing a high-volume, language-specific corpus, Python-Code-Large enables systematic experimentation in Python-focused model training, domain adaptation, and downstream… See the full description on the dataset page: https://huggingface.co/datasets/Lovett01/Python-Code-Large.Python-Code-LargePython-Code-Large
Python-Code-Large is a large-scale corpus of Python source code comprising more than 2 million rows of Python code. The dataset is designed to support research in large language model (LLM) pretraining, code intelligence, software engineering automation, and program analysis for the Python ecosystem.
By providing a high-volume, language-specific corpus, Python-Code-Large enables systematic experimentation in Python-focused model training, domain adaptation, and downstream… See the full description on the dataset page: https://huggingface.co/datasets/ajibawa-2023/Python-Code-Large.Python-Code-Solutions
Python Code Solutions
Features
1000k of Python Code Solutions for Text Generation and Question Answering
Python Coding Problems labelled by topic and difficulty
Recommendations
Train your Model on Logical Operations and Mathematical Problems Before Training it on this. This is optional for Fine Tuning 2B parameter + models.
Format the prompts in a orderly way when formatting data eg. {question} Solution: {solution} Topic: {topic}
arxiv_python_research_code
Dataset Card for "ArtifactAI/arxiv_python_research_code"
Dataset Description
https://huggingface.co/datasets/AlgorithmicResearchGroup/arxiv_python_research_code
Dataset Summary
AlgorithmicResearchGroup/arxiv_python_research_code contains over 4.13GB of source code files referenced strictly in ArXiv papers. The dataset serves as a curated dataset for Code LLMs.
How to use it
from datasets import load_dataset
# full dataset (4.13GB of data)
ds =… See the full description on the dataset page: https://huggingface.co/datasets/AlgorithmicResearchGroup/arxiv_python_research_code.python-code-instructions-85k
Python Code Instructions - 85K
Instruction-tuning dataset of Python functions paired with short natural-language instructions derived from repository docstrings.
What changed in this release
This release keeps the original public rows and format, but makes the dataset easier to use responsibly:
exact duplicate rows were removed again using normalized instruction + output hashing
deterministic train, validation, and test splits were added
the dataset card now documents… See the full description on the dataset page: https://huggingface.co/datasets/NickIBrody/python-code-instructions-85k.arxiv_deep_learning_python_research_code
ArXiv Deep Learning Python Research Code
A curated corpus of Python source code files extracted from GitHub repositories referenced in ArXiv papers. Contains 391,496 files (1.49 GB) filtered to deep learning frameworks, designed for training and evaluating Code LLMs on research-grade code.
Dataset Summary
Statistic
Value
Total files
391,496
Total size
1.49 GB
Source repos
34,099
Time span
ArXiv inception through July 2023
Dataset Structure… See the full description on the dataset page: https://huggingface.co/datasets/AlgorithmicResearchGroup/arxiv_deep_learning_python_research_code.python-code-instructions-japanese
Python Code Instructions - Japanese (18K)
Dataset Description
This dataset contains 18,612 Python programming instruction-response pairs translated to Japanese. It's designed for training language models to understand and generate Python code based on Japanese instructions.
Key Features
18,612 entries covering diverse Python programming tasks
Japanese instructions and prompts for code generation
Original English text preserved for reference
Python code… See the full description on the dataset page: https://huggingface.co/datasets/ronantakizawa/python-code-instructions-japanese.Stable-Code-Python-SFT
Stable Code Python SFT
The Stable Code Python SFT dataset is a high-quality synthetic dataset derived from the
stabilityai/stable-code-instruct-3b model for the purpose of supervised fine-tuning (SFT). Please refer to the
Versioning section for dataset versions.
Note: If you would like to contribute to this repository,
please read the CONTRIBUTING first.
TableofContents
Features
File Structure
Metadata
Usage
Versioning
License
TeamContact
Reference
Citation… See the full description on the dataset page: https://huggingface.co/datasets/bunyaminergen/Stable-Code-Python-SFT.Dr-Zeon-Github-Python-Code-Dataset
Luck Spark 1B - High Quality Code Dataset
The first quality-scored, star-agnostic code dataset for training 1B MoE code models.
Unlike The Stack / CodeParrot that filter by stars, this dataset scores every file by its content (0-10). A 2-star well-documented library scores higher than a 10k-star minified file. Continuously updated by an autonomous bot.
Repo: ahmetggg/luck-spark-1b-code-dataset | Bot: github_to_hf_bot.py | License: Permissive only (MIT / Apache-2.0 / BSD /… See the full description on the dataset page: https://huggingface.co/datasets/ahmetggg/Dr-Zeon-Github-Python-Code-Dataset.combined_coder_pythonCombining smaller python code datasets into a larger one.
Changed format to system, instruction, output.
Built from:
dataset1: nickrosh/Evol-Instruct-Code-80k-v1
dataset2: ehartford/dolphin-coder
dataset3: iamtarun/python_code_instructions_18k_alpaca
dataset4: iamtarun/python_code_instructions_18k_alpaca
dataset5: Vezora/Tested-22k-Python-Alpaca
dataset6: mlabonne/Evol-Instruct-Python-26k
dataset7: KrisPi/PythonTutor-Evol-1k-DPO-GPT4_vs_35
dataset8:… See the full description on the dataset page: https://huggingface.co/datasets/jtatman/combined_coder_python.github-python-code-fim
github python code fim
Generated from tomekkorbak/python-github-code, limiting context length to 8192.
python_code_docstring_ast_corpus
Overview
This dataset contains 34,000+ rows of code-docstring-ast data along with additional metadata. Data was gathered from various Python libraries and frameworks and their
publicly available GitHub repos. This dataset was created for the purpose of training the CodeT5+ transformer on AST-enhanced code-to-doc tasks.
Sources
The dataset was gathered from various GitHub repos sampled from this repo by Vinta.
The 26 repos are:
matplotlib
pytorch
cryptography
django… See the full description on the dataset page: https://huggingface.co/datasets/Mir-2002/python_code_docstring_ast_corpus.python_documentation_codeDataset created using https://github.com/jeffmeloy/py2dataset using the Python code from the following:
https://github.com/ansible/ansible
https://github.com/apache/airflow
https://github.com/arogozhnikov/einops
https://github.com/arviz-devs/arviz
https://github.com/astropy/astropy
https://github.com/biopython/biopython
https://github.com/bjodah/chempy
https://github.com/bokeh/bokehhttps://github.com/CalebBell/thermo
https://github.com/camDavidsonPilon/lifelines
https://github.com/coin-or/pulp… See the full description on the dataset page: https://huggingface.co/datasets/jeffmeloy/python_documentation_code.python-clean-codeparrot
python-clean-codeparrot
A cleaned, deduplicated, Python-only pretraining corpus derived from
codeparrot/codeparrot-clean,
built as the pretraining data for PocketCoder, a 95.87M-parameter decoder-only
code language model. 1,800,000 documents, ~2.96 billion tokens
(DeepSeek-Coder tokenizer, vocabulary 32,022).
Paper: PocketCoder: What Distillation, SFT, and DPO Each Buy You at 100M Parameters
Model: Ananda100/PocketCoder
SFT dataset: Ananda100/python-sft-dataset
Code:… See the full description on the dataset page: https://huggingface.co/datasets/Ananda100/python-clean-codeparrot.reason_code-search-net-python
Dataset Card for "reason_code-search-net-python"
Dataset Summary
This dataset is an instructional dataset for Python.The dataset contains five different kind of tasks.
Given a Python 3 function:
Type 1: Generate a summary explaining what it does. (For example: This function counts the number of objects stored in the jsonl file passed as input.)
Type 2: Generate a summary explaining what its input parameters represent ("For example: infile: a file descriptor of a file… See the full description on the dataset page: https://huggingface.co/datasets/Nan-Do/reason_code-search-net-python.llama-python-codes-30k
Python Codes - 30k examples, Llama1&2 tokenized dataset
Author
FlyTech
For general guide on how to create, quantize, merge or inference the model and more, visit:
hackmd.io/my_first_ai
Overview
This dataset serves as a rich resource for various Natural Language Processing tasks such as:
Question Answering
Text Generation
Text-to-Text Generation
It primarily focuses on instructional tasks in Python, tokenized specifically for the Llama architecture.… See the full description on the dataset page: https://huggingface.co/datasets/flytech/llama-python-codes-30k.python-code-cot-18k
python-code-cot-18k
COT distilled dataset with 16,565 examples.
Source
Base: iamtarun/python_code_instructions_18k_alpaca
Model: Mistral-7B-Instruct-v0.2-AWQ
Format
instruction: Task
thinking: <think>...</think> reasoning
response: Solution
ML-1M-Syntax-Validated-Python-Code
ML-1M Syntax-Validated Python Code
Dataset Summary
ML-1M Syntax-Validated Python Code is a large-scale corpus containing over 1 million machine-learning–oriented Python programs derived from The Stack, a permissively licensed collection of open-source source code.
The dataset is constructed through heuristic ML-domain filtering, syntactic validation, and basic safety checks. It is intended to support empirical analysis of real-world ML code, executability and dependency… See the full description on the dataset page: https://huggingface.co/datasets/Noushad999/ML-1M-Syntax-Validated-Python-Code.code-search-net-python
Dataset Card for "code-search-net-python"
Dataset Description
Homepage: None
Repository: https://huggingface.co/datasets/Nan-Do/code-search-net-python
Paper: None
Leaderboard: None
Point of Contact: @Nan-Do
Dataset Summary
This dataset is the Python portion of the CodeSarchNet annotated with a summary column.The code-search-net dataset includes open source functions that include comments found at GitHub.The summary is a short description of what the… See the full description on the dataset page: https://huggingface.co/datasets/khemprogrammer/code-search-net-python.Python_GOD_Coder_Omniforge_AI_12k
Python GOD Coder Omniforge AI 12k
Creator: Within Us AI
A 12,000-row mixed-format Python coding dataset designed as a sharpening corpus for building a small but dangerous Python specialist.
This dataset is intentionally focused on the practical behaviors that matter for a modern Python coding model:
implementation with tests
strict code-only instruction following
debugging and repair
refactoring for readability and production readiness
next-token code completion… See the full description on the dataset page: https://huggingface.co/datasets/WithinUsAI/Python_GOD_Coder_Omniforge_AI_12k.leetcode-codegen-python
LeetCode Code-Gen Dataset — Python
2522 rows. Given a problem statement, its input/output examples, and a
required algorithm/technique, generate a correct Python solution.
Part of a 4-language collection built from the same source: see the sibling
Python,
Java,
C++, and
JavaScript
datasets.
Verification
Every row was extracted, then executed in a sandboxed subprocess against the problem's own stated examples. Only rows that passed all examples are included -- this… See the full description on the dataset page: https://huggingface.co/datasets/AmareshHebbar/leetcode-codegen-python.python-github-code-instruct-filtered-5k
Dataset Card for "python-github-code-instruct-filtered-5k"
This fine dataset tomekkorbak/python-github-code, filtered by scores greater than 0.03.
Feedback and additional columns generated through OpenAI and Cohere responses.
Instruct-Python-Code-Turkish
Dataset Card for Instruct-Python-Code-Turkish
Language: Turkish
Dataset Description
The translation was performed using the Google translation model to ensure high-quality, accurate translation.
Dataset Details
Size: ≈5K
Translation tool: Google Translate
Data format: Instruct, Output
python-codes-25k
License
MIT
This is a Cleaned Python Dataset Covering 25,000 Instructional Tasks
Overview
The dataset has 4 key features (fields): instruction, input, output, and text.It's a rich source for Python codes, tasks, and extends into behavioral aspects.
Dataset Statistics
Total Entries: 24,813
Unique Instructions: 24,580
Unique Inputs: 3,666
Unique Outputs: 24,581
Unique Texts: 24,813
Average Tokens per example: 508
Features… See the full description on the dataset page: https://huggingface.co/datasets/xphillyx/python-codes-25k.