datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
SWE-smith
SWE-smith Dataset
Code
•
Paper
•
Site
[12/14/2025] NOTE: We will no longer actively update this dataset.
While this dataset is still functional and usable, we recommend you use the `SWE-bench/SWE-smith-[lang]` datasets.
For better maintainability and ease-of-use, we are maintaining language-specific datasets in lieu of this mono-repo.
The SWE-smith Dataset is a training dataset of 50137 task instances from 128 GitHub repositories, collected using the SWE-smith toolkit.… See the full description on the dataset page: https://huggingface.co/datasets/SWE-bench/SWE-smith.SWE-smith-py
SWE-smith Dataset
Code
•
Paper
•
Site
As of 12/14/2025, SWE-smith: Python contains 50908 task instances from 131 GitHub repositories
The SWE-smith Dataset is the largest open source dataset for training software engineering agents.
All SWE-smith task instances come with an executable environment.
To learn more about how to use this dataset to train Language Models for Software Engineering, please refer to the documentation.
SWE-smith-pythonSWE-smith-javaSWE-smith2SWE-smith-trajectories
SWE-smith Trajectories
Code
•
Paper
•
Site
This dataset contains the 5017 trajectories we fine-tuned Qwen 2.5 Coder Instruct on, leading to
SWE-agent-LM-32B, a coding LM agent that
achieve 40.2% on SWE-bench Verified (no verifiers or multiple rollouts, just 1 attempt per instance).
Trajectories were generated by running SWE-agent + Claude 3.7 Sonnet on task instances from
the SWE-smith dataset.
SWE-smith-jsMulti-SWE-smith-tasksSWE-smith-tsminty-astro-ph
MINT-1T ArXiv Astro-ph
An astronomy-focused subset of mlfoundations/MINT-1T-ArXiv, filtered to include only papers from the astro-ph arXiv category (including cross-listed papers).
Overview
Papers
~845k
Total size
~804 GB
Format
WebDataset tar shards
Shards
287 (astro-ph-00000.tar to astro-ph-00286.tar)
Shard size
~3 GB each
Source
MINT-1T (Awadalla et al., 2024)
Data Format
Each tar shard contains paired files per paper:… See the full description on the dataset page: https://huggingface.co/datasets/Smith42/minty-astro-ph.SWE-smith-cppSWE-smith-go
SWE-smith Dataset
Code
•
Paper
•
Site
As of 12/14/202, SWE-smith: Golang contains 8212 task instances from 87 GitHub repositories
The SWE-smith Dataset is the largest open source dataset for training software engineering agents.
All SWE-smith task instances come with an executable environment.
To learn more about how to use this dataset to train Language Models for Software Engineering, please refer to the documentation.
galaxies
Galaxies for training astroPT
Here we have ~8.5 million galaxy cutouts from the DESI legacy survey DR8.
The cut outs are 512x512 pixel jpg images centred on the galaxy source.
I've split away 1% of the images into a test set, and 1% into a validation set.
The remaining 98% of the images comprise the training set.
Metadata (galaxy properties)
There is also accompanying metadata!
To combine the metadata with the galaxy images you can do (for example):
from datasets import… See the full description on the dataset page: https://huggingface.co/datasets/Smith42/galaxies.SWE-smith-mini_swe_agent_plus-trajectories-66k
Dataset: SWE-smith-mini_swe_agent_plus-trajectories-66k
A corpus of ~66k issue-solving trajectories collected with mini-swe-agent-plus on issues derived from SWE-smith. Each trajectory records the agent’s end-to-end process.
We training the Qwen3-8B model on different sizes of the training data. The results are shown in the figure, it could be observed that the solve rate on SWE-bench Verified improves approximately linearly with the logarithm of the data scale (1k → 66k… See the full description on the dataset page: https://huggingface.co/datasets/Kwai-Klear/SWE-smith-mini_swe_agent_plus-trajectories-66k.smiles-transformers
smiles-transformers dataset
TODO: Add references to the datasets we curated
dataset features
name: text
Molecule SMILES : string
name: formula
Molecular formula : string
name: NumHDonors
Number of hidrogen bond donors : int
name: NumHAcceptors
Number of hidrogen bond acceptors : int
name: MolLogP
Wildman-Crippen LogP : float
name: NumHeteroatoms
Number of hetero atoms: int
name: RingCount
Number of rings : int
name: NumRotatableBonds
Number of rotable… See the full description on the dataset page: https://huggingface.co/datasets/maykcaldas/smiles-transformers.SWE-smith-oracleThis is a version of SWE-bench/SWE-smith filtered for non-empty problem_statement and formatted into the oracle setting of SWE-bench where the files edited by the patch are displayed to the agent. This problem presentation is made available in a text column, following the format of princeton-nlp/SWE-bench_Lite_oracle.
SWE-smith-rssmithsonian_butterflies_subsetThis a subset of "ceyda/smithsonian_butterflies" dataset with additional processing done to train the "ceyda/butterfly_gan" model.
The preprocessing includes:
Adding "sim_score" to images with CLIP model using "pretty butterfly","one butterfly","butterfly with open wings","colorful butterfly"
Removing butterflies with the same name(species)
Limiting only to the top 1000 images
Removing the background (doing another sim_scoring after bg removal did visually worse so didn't do it)
Detecting… See the full description on the dataset page: https://huggingface.co/datasets/huggan/smithsonian_butterflies_subset.SWE-smith-phpSWE-smith-filteredFiltered dataset from SWE-bench/SWE-smith that the problem_statement is not empty for quick loading.
After removing the rows with empty problem_statement, it comes to 52945 -> 10924 rows (removed 42021 empty problem_statements)
SWE-smith-trajectories-harborimport datasets
def map_to_harbor(example):
# in image name, change jyangballin/ for swesmith/
example['image_name'] = example['image_name'].replace('jyangballin/', 'swesmith/')
return example
smith = datasets.load_dataset('SWE-bench/SWE-smith', split='train')
smith = smith.map(map_to_harbor)
smith = datasets.DatasetDict({'train': smith})
smith.push_to_hub('ricdomolm/SWE-smith-trajectories-harbor')
SWE-smith-js-4259-filteredSWE-smith-filteredFiltered SWE-smith instances that have problem statements as of May 23rd, 2025. Also maximally compressed with categorical variables for low cardinality values.
SWE-smith-jsSWE-smith-java-6704-filtered-for-problem-statementsswe_smith_js_5902_filteredSWE-smith-py-39471-filtered-for-problem-statementsPubChem-124M-SMILES-SELFIES-InChI-IUPAC
PubChem-124M-Canonicalized-SELFIES-InChI-IUPAC
Dataset Summary
This dataset contains ~124 million chemical structures sourced from PubChem (as of Jan 2026), processed into a clean, machine-learning-ready Parquet format.
Unlike raw XML/JSON dumps or standard CSVs, this dataset provides a unified, tabular structure that joins multiple chemical identifiers and descriptors into a single sharded resource:
SMILES: Raw and RDKit-Canonicalized.
SELFIES: Pre-computed 100% robust… See the full description on the dataset page: https://huggingface.co/datasets/hheiden/PubChem-124M-SMILES-SELFIES-InChI-IUPAC.swe_smith_back_translationBack translate the swe-smith data to get the problem statment following the R2E sylte prompt. More details in https://github.com/SWE-bench/SWE-smith/issues/127
kor_unsmile