datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
librispeech_asr_dummyzentesting_alpaca_small
Dataset Card for "testing_alpaca_small"
More Information needed
testing_self_instruct_small
Dataset Card for "testing_self_instruct_small"
More Information needed
zen-imagegneissweb-annotation-url-testing-v1
GneissWeb Annotations
GneissWeb Annotations, powered by IBM Research's GneissWeb methodology, is a dataset of quality and category annotations applied to the Common Crawl corpus.
This dataset enables precise filtering of web content across medical, educational, technology, and scientific domains, making it easier to build high-quality corpora for research projects, language models, and specialized applications.
Learn more about the annotation process and methodology in our… See the full description on the dataset page: https://huggingface.co/datasets/commoncrawl/gneissweb-annotation-url-testing-v1.dummy_image_text_data
Dataset Card for "dummy_image_text_data"
More Information needed
testing_codealpaca_small
Dataset Card for "testing_codealpaca_small"
More Information needed
harmonytoolcallzen-multi-imagehost-index-testing-v2
Common Crawl Host Index v2
GitHub: https://github.com/commoncrawl/cc-host-index
Each crawl, we generate a Host Index, which aggregates information about each web hosted visited during the crawl. The
information is aggregated from the Common Crawl columnar index,
web graph, and raw crawler logs.
Quickstart
The dataset is Hive-partitioned on crawl (data/crawl=CC-MAIN-2025-18/*.parquet). Open the whole
dataset once, then filter with WHERE crawl = '...': because… See the full description on the dataset page: https://huggingface.co/datasets/commoncrawl/host-index-testing-v2.fixtures_docvqaThis dataset includes 2 document images of the DocVQA dataset.
They are used for testing the LayoutLMv2FeatureExtractor + LayoutLMv2Processor inside the HuggingFace Transformers library.
More specifically, they are used in tests/test_feature_extraction_layoutlmv2.py and tests/test_processor_layoutlmv2.py.
librispeech_asr_demodummy-base64-imageswiki_dpr_dummyThis dummy dataset is used for testing purpose for rag model in transformers. It is proudced via the following steps:
dataset = datasets.load_dataset("wiki_dpr", with_embeddings=True, with_index=True, index_name="exact", embeddings_name="nq", dummy=True, revision=None)
dataset["train"].drop_index("embeddings")
dataset.push_to_hub("hf-internal-testing/wiki_dpr_dummy", token="...")
The index file `index.faiss` (after being renamed locally) is then uploaded manually.
NanoBEIR-defill10descriptiveness-sentiment-trl-style
TRL's Sentiment and Descriptiveness Preference Dataset
The dataset comes from https://arxiv.org/abs/1909.08593, one of the earliest RLHF work from OpenAI.
We preprocess the dataset using our standard prompt, chosen, rejected format.
Reproduce this dataset
Download the descriptiveness_sentiment.py from the https://huggingface.co/datasets/trl-internal-testing/descriptiveness-sentiment-trl-style/tree/0.1.0.
Run python examples/datasets/descriptiveness_sentiment.py… See the full description on the dataset page: https://huggingface.co/datasets/trl-internal-testing/descriptiveness-sentiment-trl-style.stress_testing_model_spec
Stress Testing Model Spec Compliance
A dataset of AI model responses to queries designed to create value tensions, along with compliance evaluations from multiple judge models.
Configurations
1. Subset Default (6 splits)
dataset = load_dataset('jifanz/stress_testing_model_spec')
Six curated subsets of query-response pairs.
Columns:
rank: Integer ranking
query_generator: Which model generated the query (e.g., "o3", "claude_opus_4")
value1, value2: Two values… See the full description on the dataset page: https://huggingface.co/datasets/jifanz/stress_testing_model_spec.dolly-chatml-sftinstructpix2pix-10-samples
Dataset Card for "test"
More Information needed
abc-testing
ABC Testing dataset
A: A0-2**24
B: B0-2**20
C: C0-2**16
gneissweb-annotation-host-testing-v1
GneissWeb Annotations
GneissWeb Annotations, powered by IBM Research's GneissWeb methodology, is a dataset of quality and category annotations applied to the Common Crawl corpus.
This dataset enables precise filtering of web content across medical, educational, technology, and scientific domains, making it easier to build high-quality corpora for research projects, language models, and specialized applications.
Learn more about the annotation process and methodology in our… See the full description on the dataset page: https://huggingface.co/datasets/commoncrawl/gneissweb-annotation-host-testing-v1.arc_agi_3_public_demo_human_testing
Dataset Card for ARC-AGI 3 Public Demo Human Testing
Dataset Summary
This dataset contains human gameplay logs and trajectories from the ARC-AGI 3 public demo. It is a fully open-source dataset created by the ARC Prize.
The primary purpose of publishing this dataset on Hugging Face is to make it easily accessible and convenient for participants in the Kaggle ARC Prize 2026 Competition.
The implementation and source code used to process and upload this dataset to… See the full description on the dataset page: https://huggingface.co/datasets/magic-sword/arc_agi_3_public_demo_human_testing.hh-rlhf-helpful-base-trl-style
TRL's Anthropic HH Dataset
We preprocess the dataset using our standard prompt, chosen, rejected format.
Reproduce this dataset
Download the anthropic_hh.py from the https://huggingface.co/datasets/trl-internal-testing/hh-rlhf-helpful-base-trl-style/tree/0.1.0.
Run python examples/datasets/anthropic_hh.py --push_to_hub --hf_entity trl-internal-testing
document-visual-retrieval-test
Model Card: Document Visual Retrieval Test (internal)
Dataset Overview
This dataset is designed to evaluate the performance of visual retrievers by testing their ability to match a query to a relevant image. Each of the three examples in this dataset contains a text query and an associated image, which is a scanned page from the foundational "Attention is All You Need" paper. The purpose of this dataset is to facilitate the evaluation of visual retrievers, where the… See the full description on the dataset page: https://huggingface.co/datasets/hf-internal-testing/document-visual-retrieval-test.dailytalk-dummycode_x_glue_cc_cloze_testing_all
Dataset Card for "code_x_glue_cc_cloze_testing_all"
Dataset Summary
CodeXGLUE ClozeTesting-all dataset, available at https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/ClozeTesting-all
Cloze tests are widely adopted in Natural Languages Processing to evaluate the performance of the trained language models. The task is aimed to predict the answers for the blank with the context of the blank, which can be formulated as a multi-choice classification problem.
Here we… See the full description on the dataset page: https://huggingface.co/datasets/google/code_x_glue_cc_cloze_testing_all.penetration_testing_scraped_dataset
Dataset Card for "penetration_testing_scraped_dataset"
More Information needed