datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
the-stack-dedup-python-filtered-docstringsThis is a dataset originated from bigcode/the-stack-dedup with some filters applied.
The filters filtered in this dataset are:
remove_function_no_docstring
remove_class_no_docstring
remove_delete_markers
the-stack-dedup-python-filtered-docstrings-gpt2text-code-galeras-code-generation-from-docstring-3k-dedupedthe-stack-smol-python-docstrings
Dataset Card for "the-stack-smol-filtered-python-docstrings"
More Information needed
code_docstring_corpusHF version of Edinburgh-NLP's Code docstrings corpus
python_code_docstring_ast_corpus
Overview
This dataset contains 34,000+ rows of code-docstring-ast data along with additional metadata. Data was gathered from various Python libraries and frameworks and their
publicly available GitHub repos. This dataset was created for the purpose of training the CodeT5+ transformer on AST-enhanced code-to-doc tasks.
Sources
The dataset was gathered from various GitHub repos sampled from this repo by Vinta.
The 26 repos are:
matplotlib
pytorch
cryptography
django… See the full description on the dataset page: https://huggingface.co/datasets/Mir-2002/python_code_docstring_ast_corpus.arkts-code-docstring-pro
ArkTS-CodeSearch-Pro: A Cleaned and License-Aware ArkTS Dataset
Original Dataset | Pro Dataset | Paper | Evaluation Code | Dataset Processing Code | Fine-Tuned Model
This dataset is an improved version of ArkTS-CodeSearch. It contains function-level information from ArkTS (.ets) projects, including function source code, docstrings, AST representations, and repository metadata. It is intended for code retrieval, code understanding, and AST-based research.
Compared with the… See the full description on the dataset page: https://huggingface.co/datasets/hreyulog/arkts-code-docstring-pro.thevault-docstringstyle
Dataset Card for "thevault-docstringstyle"
More Information needed
arkts-code-docstring
Recommended Version: ArkTS-CodeSearch-Pro
We strongly recommend using ArkTS-CodeSearch-Pro for all new research and development.The Pro version is the actively maintained release and provides improved data quality through stricter open-source license filtering, repository quality control (stars > 0), comment normalization, removal of license/copyright headers and code-only comments, minimum docstring length filtering, and global code/query deduplication.
The original… See the full description on the dataset page: https://huggingface.co/datasets/hreyulog/arkts-code-docstring.29K_Python_Docstring_Pairs
29K High-Quality Python Docstring Pairs
Author: Michael Hernandez (XxCotHGxX)License: CC BY 4.0Cleaned from: XxCotHGxX/242K_Python_Docstring_Pairs
Overview
A curated, high-quality subset of Python function–docstring pairs for use in code documentation generation, docstring completion, and code understanding tasks.
The original 242K dataset was scraped from open-source Python repositories but contained a significant proportion of functions without docstrings (84% of… See the full description on the dataset page: https://huggingface.co/datasets/XxCotHGxX/29K_Python_Docstring_Pairs.stack-smol-docstrings
Stack-Smol-Docstrings
This dataset contains Python functions extracted from the-stack-smol,
filtered for high-quality docstrings and implementations. Each sample includes
the function's docstring, implementation, and a masked version of the code where the
function is replaced with a comment.
The dataset is designed for code completion tasks where a model needs to restore a
function that has been replaced with a comment. The model is provided with:
The full file context with the… See the full description on the dataset page: https://huggingface.co/datasets/ASSERT-KTH/stack-smol-docstrings.arkts-code-docstring-for-anonymous
ArkTS-CodeSearch-Pro: A Cleaned and License-Aware ArkTS Dataset
Evaluation Code |Dataset processing Code | Fine-Tuned Model
This dataset contains function-level information from ArkTS (.ets) projects, including function source code, normalized docstrings, AST representations, and repository metadata. It is intended for code retrieval, code understanding, and AST-based research.
Compared with the original release, this Pro version provides the following improvements:… See the full description on the dataset page: https://huggingface.co/datasets/hreyulog/arkts-code-docstring-for-anonymous.the-stack-dedup-python-docstrings-1.0-percent-unified
Dataset Card for "the-stack-dedup-python-docstrings-1.0-percent-unified"
More Information needed
mined_docstrings_pypiStats:
total 352M
149228 drwxr-xr-x 2 micha micha 324K May 12 11:13 .
134775 drwxr-xr-x 4 micha micha 4.0K May 12 11:13 ..
168137 -rw-r--r-- 1 root root 6.2K May 12 11:06 2dwavesim.parquet
159971 -rw-r--r-- 1 root root 3.0K May 12 08:19 aad_token_verify.parquet
164105 -rw-r--r-- 1 root root 3.6K May 12 08:51 aadict.parquet
165435 -rw-r--r-- 1 root root 437 May 12 09:36 aats.parquet
166392 -rw-r--r-- 1 root root 437 May 12 10:13 abarms.parquet
158836 -rw-r--r-- 1 root root… See the full description on the dataset page: https://huggingface.co/datasets/michaelfeil/mined_docstrings_pypi.python_googlestyle_docstringsmined_docstrings_pypi_embeddedDataset embedded with jina-code-v2
python-docstring-human-gpt-generated-mixcode-code-galeras-code-completion-from-docstring-3k-dedupedpython-docstrings
Python Docstring Diff Dataset
This dataset contains training samples for models that generate Python documentation patches.
Each example provides a Python source file with its docstrings removed and a corresponding unified diff patch that restores the documentation.
The dataset is designed for training or evaluating language models that assist with:
Automatic code documentation
Docstring generation
Code review automation
Developer tooling
Dataset Structure
Each entry contains the… See the full description on the dataset page: https://huggingface.co/datasets/helloadhavan/python-docstrings.python-google-style-docstrings
Overview
This dataset contains Python code-docstring pairs, whereas the docstrings are in Google style. A Google style docstring is structured as follows:
<Description of the code>
Args:
<var1> (<data-type>) : <description of var1>
<var2> (<data_type>) : <description of var2>
Returns:
<var3> (<data-type>) : <description of var3>
Raises:
<var4> (<data-type>) : <description of var4>
The format varies widely (like additional sections such as Examples, Notes, etc) but generally… See the full description on the dataset page: https://huggingface.co/datasets/Mir-2002/python-google-style-docstrings.code_docstringsCodeXGLUE-Code-Docstringcode-docstring-datasetCodeXGLUE-AST-Docstringworkshop-sbt-docstringstack-v2-cpp-with-docstring-dataset_v1vllm_add_docstring_datasetstyle_complexity_docstring_1760style_complexity_docstring_v2_2232juraj-juraj-python-docstring-human-gpt-generated-mix