datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
wikitext
Dataset Card for "wikitext"
Dataset Summary
The WikiText language modeling dataset is a collection of over 100 million tokens extracted from the set of verified
Good and Featured articles on Wikipedia. The dataset is available under the Creative Commons Attribution-ShareAlike License.
Compared to the preprocessed version of Penn Treebank (PTB), WikiText-2 is over 2 times larger and WikiText-103 is over
110 times larger. The WikiText dataset also features a far… See the full description on the dataset page: https://huggingface.co/datasets/Salesforce/wikitext.OpenThoughts-1k-sample
[!NOTE]
We have released a paper for OpenThoughts! See our paper here.
Open-Thoughts-1k-sample
This is a 1k sample of the OpenThoughts-114k dataset.
Open synthetic reasoning dataset with high-quality examples covering math, science, code, and puzzles!
Inspect the content with rich formatting with Curator Viewer.
Available Subsets
default subset containing ready-to-train data used to finetune the OpenThinker-7B and OpenThinker-32B models:
ds =… See the full description on the dataset page: https://huggingface.co/datasets/ryanmarten/OpenThoughts-1k-sample.PhysicalAI-Robotics-GR00T-X-Embodiment-Sim
PhysicalAI-Robotics-GR00T-X-Embodiment-Sim
Github Repo: Isaac GR00T N1
We provide a set of datasets used for post-training of GR00T N1. Each dataset is a collection of trajectories from different robot embodiments and tasks.
Cross-embodied bimanual manipulation: 9k trajectories
Dataset Name
#trajectories
bimanual_panda_gripper.Threading
1000
bimanual_panda_hand.LiftTray
1000
bimanual_panda_gripper.ThreePieceAssembly
1000… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-GR00T-X-Embodiment-Sim.many-peptides-md
[!IMPORTANT]
Critical Update
The original 8AA TICA models within subsampled_trajectories/*/8AA/*.npz employed a CA-only atom selection. These models are not valid for comparison to results in our paper.
Updated files (uploaded 15/12/2025) now contain corrected models. If you previously downloaded this dataset, please re-download to ensure accurate results.
Note: Codebase references to tica_features_ca must now be replaced with tica_features. This was resolved in our codebase by PR #26.
Note:… See the full description on the dataset page: https://huggingface.co/datasets/transferable-samplers/many-peptides-md.standard-chess-games
[!CAUTION]
This dataset is still a work in progress and some breaking changes might occur.
Lichess Rated Standard Chess Games Dataset
Dataset Description
6,771,826,271 standard rated games, played on lichess.org, updated monthly from the database dumps.
This version of the data is meant for data analysis. If you need PGN files you can find those here. That said, once you have a subset of interest, it is trivial to convert it back to PGN as shown in the Dataset Usage… See the full description on the dataset page: https://huggingface.co/datasets/Lichess/standard-chess-games.super_glue
Dataset Card for "super_glue"
Dataset Summary
SuperGLUE (https://super.gluebenchmark.com/) is a new benchmark styled after
GLUE with a new set of more difficult language understanding tasks, improved
resources, and a new public leaderboard.
Supported Tasks and Leaderboards
More Information Needed
Languages
More Information Needed
Dataset Structure
Data Instances
axb
Size of downloaded dataset files: 0.03 MB
Size of… See the full description on the dataset page: https://huggingface.co/datasets/aps/super_glue.preprocessed_commoncatalog-cc-byI also seperately provide just the prompts in prompts.json
keys are the image_id, and the values are the captions generated
Captions generated by moondream: vikhyatk/moondream2
Latents generated by SDXL VAE: madebyollin/sdxl-vae-fp16-fix
Embeddings generated by SigLIP: hf-hub:timm/ViT-SO400M-14-SigLIP-384
Original dataset: common-canvas/commoncatalog-cc-by
Latents f32 and embeddings are f16 bytes
Compute cost: 16x3090 for 3 day. Approximately.
physics-course-vidsshShareGPT_Vicuna_unfilteredFurther cleaning done. Please look through the dataset and ensure that I didn't miss anything.
Update: Confirmed working method for training the model: https://huggingface.co/AlekseyKorshuk/vicuna-7b/discussions/4#64346c08ef6d5abefe42c12c
Two choices:
Removes instances of "I'm sorry, but": https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/blob/main/ShareGPT_V3_unfiltered_cleaned_split_no_imsorry.json
Has instances of "I'm sorry, but":… See the full description on the dataset page: https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered.gaiaThis catalog is developed for use with the Siril 1.4 series as a public reference database. Hugging Face is one of several mirrors used to distribute the data. This database is provided for both offline download and also for online access. This dataset is provided for scientific and reproducibility purposes.
This is an extract of the Gaia DR3 catalog optimized for spectrophotometric color calibration. The catalog is indexed at HEALpix level 8 and selects up to the 127 brightest sources in each… See the full description on the dataset page: https://huggingface.co/datasets/siril-spcc/gaia.cc
What is Symato CC?
To download all WARC data from Common Crawl then filter out Vietnamese in Markdown and Plaintext format.
There is 1% of Vietnamse in CC, extract all of them out should be a lot (~10TB of plaintext).
Main contributors
https://huggingface.co/nampdn-ai
https://huggingface.co/binhvq
https://huggingface.co/th1nhng0
https://huggingface.co/iambestfeed
Simple quality filters
To make use of raw data from common crawl, you need to do filtering… See the full description on the dataset page: https://huggingface.co/datasets/Symato/cc.SAGE-10k
SAGE-10k
SAGE-10k is a large-scale interactive indoor scene dataset featuring realistic layouts, generated by the agentic-driven pipeline introduced in "SAGE: Scalable Agentic 3D Scene Generation for Embodied AI". The dataset contains 10,000 diverse scenes spanning 50 room types and styles, along with 565K uniquely generated 3D objects.
🔑 Key Features
SAGE-10k integrates a wide variety of scenes, and particularly, preserves small items… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/SAGE-10k.GiftEvalPretrain
GIFT-Eval Pre-training Datasets
Pretraining dataset aligned with GIFT-Eval that has 71 univariate and 17 multivariate datasets, spanning seven domains and 13 frequencies, totaling 4.5 million time series and 230 billion data points. Notably this collection of data has no leakage issue with the train/test split and can be used to pretrain foundation models that can be fairly evaluated on GIFT-Eval.
📄 Paper
🖥️ Code
📔 Blog Post
🏎️ Leader Board
Ethical Considerations… See the full description on the dataset page: https://huggingface.co/datasets/Salesforce/GiftEvalPretrain.sciq
Dataset Card for "sciq"
Dataset Summary
The SciQ dataset contains 13,679 crowdsourced science exam questions about Physics, Chemistry and Biology, among others. The questions are in multiple-choice format with 4 answer options each. For the majority of the questions, an additional paragraph with supporting evidence for the correct answer is provided.
Supported Tasks and Leaderboards
More Information Needed
Languages
More Information Needed… See the full description on the dataset page: https://huggingface.co/datasets/allenai/sciq.Syn4D
Syn4D: A Multiview Synthetic 4D Dataset
Syn4D is a synthetic 4D dataset with multi-view RGB videos, depth, masks, tracking geometry, and supporting object mesh metadata.
Layout
data/
syn4d_v1_stride_1/ # Syn4D V1, every frame can be a tracking reference frame
syn4d_v1_stride_1_attachments/ # Additional per-scene attachments for Syn4D V1 stride-1
syn4d_v1_stride_5/ # Syn4D V1, every 5th frame can be a tracking… See the full description on the dataset page: https://huggingface.co/datasets/Syn4D/Syn4D.nonmyopia_resultscybergymrdpsquad
Dataset Card for SQuAD
Dataset Summary
Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
SQuAD 1.1 contains 100,000+ question-answer pairs on 500+ articles.
Supported Tasks and Leaderboards
Question… See the full description on the dataset page: https://huggingface.co/datasets/rajpurkar/squad.pkgsrchttps://github.com/stal-ix/stal-ix.github.io/blob/main/MIRROR.md
met-office-uk-deterministic-solar
Met Office UK Deterministic Dataset (Zarr Format)
Description
This dataset is a subset of the Met Office UK Deterministic Dataset, converted from the original NetCDF format into Zarr format for modern data analysis. The Zarr files are packaged as .zarr.zip archives for efficient storage and transfer.
The subset focuses on specific variables and configurations, which are detailed below. Researchers and developers can use this subset for applications in climate science… See the full description on the dataset page: https://huggingface.co/datasets/openclimatefix/met-office-uk-deterministic-solar.SWE-bench_VerifiedDataset Summary
SWE-bench Verified is a subset of 500 samples from the SWE-bench test set, which have been human-validated for quality. SWE-bench is a dataset that tests systems’ ability to solve GitHub issues automatically. See this post for more details on the human-validation process.
The dataset collects 500 test Issue-Pull Request pairs from popular Python repositories. Evaluation is performed by unit test verification using post-PR behavior as the reference solution.
The original… See the full description on the dataset page: https://huggingface.co/datasets/princeton-nlp/SWE-bench_Verified.ngii-map-full-light
ngii-map-full-light
Light point/line extract from NGII 1/1000 topographic data for Korea.
Not for shipping into GitHub — use this Hugging Face dataset instead.
CRS
Korea_2000_Central_Belt_2010 projected meters [x, y]
Layers (per region under by_region/<region>/)
Layer
Description
C023
poles (전주/통신주)
C022
lights (가로등·보안등)
A002
roads (도로 중심선)
B001_tiny
building footprints <25 m² as centroids
B002
lines (구분/재질 라인)
Also:… See the full description on the dataset page: https://huggingface.co/datasets/SKPark1/ngii-map-full-light.genshin-voices-separatedstereo-550Stereo-550
Paper ·
Code ·
Build it yourself ·
3D viewer ·
Blog
Collected with FPV Labs Open-Source Stereo Hardware
Dataset overview
A first-person calibrated stereo RGB video dataset capturing everyday human manipulation across objects, materials, tools, and multi-step activities. Every session is recorded as a synchronized left/right camera pair with per-session stereo calibration, giving the visual geometry of hands, object interaction, state… See the full description on the dataset page: https://huggingface.co/datasets/fpvlabs/stereo-550.SWE-smith
SWE-smith Dataset
Code
•
Paper
•
Site
[12/14/2025] NOTE: We will no longer actively update this dataset.
While this dataset is still functional and usable, we recommend you use the `SWE-bench/SWE-smith-[lang]` datasets.
For better maintainability and ease-of-use, we are maintaining language-specific datasets in lieu of this mono-repo.
The SWE-smith Dataset is a training dataset of 50137 task instances from 128 GitHub repositories, collected using the SWE-smith toolkit.… See the full description on the dataset page: https://huggingface.co/datasets/SWE-bench/SWE-smith.gpic
GPIC: A Giant Permissive Image Corpus for Visual Generation
Keshigeyan Chandrasegaran*1,
Kyle Sargent*1,
Suchir Agarwal1,
Michael Jang1,
Michael Poli1,2,
Juan Carlos Niebles1,4,
Justin Johnson3,
Jiajun Wu1,
Li Fei-Fei1
1 Stanford University
2 Radical Numerics
3 University of Michigan
4 Salesforce… See the full description on the dataset page: https://huggingface.co/datasets/stanford-vision-lab/gpic.imdb
Dataset Card for "imdb"
Dataset Summary
Large Movie Review Dataset.
This is a dataset for binary sentiment classification containing substantially more data than previous benchmark datasets. We provide a set of 25,000 highly polar movie reviews for training, and 25,000 for testing. There is additional unlabeled data for use as well.
Supported Tasks and Leaderboards
More Information Needed
Languages
More Information Needed
Dataset Structure… See the full description on the dataset page: https://huggingface.co/datasets/stanfordnlp/imdb.DEH-image-scan-data