datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
mrcr
OpenAI MRCR: Long context multiple needle in a haystack benchmark
OpenAI MRCR (Multi-round co-reference resolution) is a long context dataset for benchmarking an LLM's ability to distinguish between multiple needles hidden in context.
This eval is inspired by the MRCR eval first introduced by Gemini (https://arxiv.org/pdf/2409.12640v2). OpenAI MRCR expands the tasks's difficulty and provides opensource data for reproducing results.
The task is as follows: The model is given a long… See the full description on the dataset page: https://huggingface.co/datasets/openai/mrcr.openai-moderation-api-evaluation
Evaluation dataset for the paper "A Holistic Approach to Undesired Content Detection"
The evaluation dataset data/samples-1680.jsonl.gz is the test set used in this paper.
Each line contains information about one sample in a JSON object and each sample is labeled according to our taxonomy. The category label is a binary flag, but if it does not include in the JSON, it means we do not know the label.
Category
Label
Definition
sexual
S
Content meant to arouse sexual… See the full description on the dataset page: https://huggingface.co/datasets/mmathys/openai-moderation-api-evaluation.genebench-pro-public-package
GeneBench-Pro Public Case Studies
This repository contains public GeneBench-Pro case studies. It is the
self-contained package intended for public distribution, including Hugging Face
publication.
Package Layout
<repo-root>/
├── .gitattributes
├── README.md
├── LICENSE
├── problems.csv
├── checksums.sha256
├── manifest.json
├── reference_definitions.md
├── reference_grader.py
└── problems/
└── <eval_id>/
├── eval_config.json
├── data_files/… See the full description on the dataset page: https://huggingface.co/datasets/openai/genebench-pro-public-package.Compact_OpenAIRE_citation_graph
📚 Compact OpenAIRE Citation Graph
Based on OpenAIRE Graph v11.1.1 (source on Zenodo).
The complete OpenAIRE citation graph, distilled into a handful of compact, analysis-ready files — the full scholarly citation network of the open-science ecosystem, small enough to actually work with.
Citation graphs at this scale are usually locked behind multi-terabyte dumps and heavyweight infrastructure. This dataset makes the entire OpenAIRE citation network loadable… See the full description on the dataset page: https://huggingface.co/datasets/Zmeos/Compact_OpenAIRE_citation_graph.webgpt_comparisonsWebGPT Comparisons contains all of the comparisons marked as suitable for reward modelling from the WebGPT paper.coval
CoVal
Public input on model behavior
Summary. CoVal (crowd-originated, values-aware preferences and rubrics) is a human-feedback dataset focused on value-sensitive model behavior. It has three components: two conversation-level files and one annotator-level file. The first conversation-level file contains (i) a synthetic prompt represented as a minimal chat transcript, (ii) four candidate assistant responses, and (iii) annotator assessments with rationales. The second… See the full description on the dataset page: https://huggingface.co/datasets/openai/coval.cc12m_openai_clip-vit-base-patch32_image_image_retrieval_pairs_2022-09-13openai-moderation-dataset
Evaluation dataset for the paper "A Holistic Approach to Undesired Content Detection"
The evaluation dataset is the test set used in this paper.
Each line contains information about one sample in a JSON object and each sample is labeled according to our taxonomy. The category label is a binary flag, but if it does not include in the JSON, it means we do not know the label.
Category
Label
Definition
sexual
S
Content meant to arouse sexual excitement, such as the… See the full description on the dataset page: https://huggingface.co/datasets/walledai/openai-moderation-dataset.openai-news_beirThis is a copy of https://huggingface.co/datasets/jinaai/openai-news reformatted into the BEIR format. For any further information like license, please refer to the original dataset.
Disclaimer
This dataset may contain publicly available images or text data. All data is provided for research and educational purposes only. If you are the rights holder of any content and have concerns regarding intellectual property or copyright, please contact us at "support-data (at) jina.ai" for… See the full description on the dataset page: https://huggingface.co/datasets/jinaai/openai-news_beir.wikipedia-1k-cohere-openai-embeddingsSmaller version of https://huggingface.co/datasets/Cohere/wikipedia-22-12-simple-embeddings that includes Cohere as well as OpenAI embeddings (text-embedding-ada-002)
100k version of this dataset will be released soon.
mscoco_train_2014_openai_clip-vit-base-patch32_image_image_retrieval_pairs_2022-09-13cc12m_openai_clip-vit-base-patch32_image_image_retrieval_pairs_2022-09-15OpenAI-MRCR-Translation-JPN本データセットは、ロングコンテキス評価データセット OpenAI MRCR を翻訳して作成した、日本語版の MRCR の評価データセットです。
翻訳方法
日本語版の作成にあたっては、元の英語データの構造を維持しつつ、LLM（Qwen3-235B-A22B）を用いて日本語版のテキストを生成しました。
作成方法としては、
会話履歴を分解
パーツごとに翻訳
元の順番に会話を並べなおす
といった手順により、日本語版の評価サンプルを作成しました。
翻訳が必要だったのは主にタスクの説明文、ユーザの問い合わせ文章、ユーザの最終問い合わせ文章の3箇所です。
タスクの説明文については固定のプロンプトなので、プロンプト全体を一度だけ翻訳しました。
ユーザの問い合わせ文については、全てのユーザの問い合わせが「write a (Document-Type) about (Genre)」という形式の英文になっていたため、Document-Type, Genre の位置の語句を抜き出して翻訳し「(Genre) についての (Document-Type)… See the full description on the dataset page: https://huggingface.co/datasets/abeja/OpenAI-MRCR-Translation-JPN.swe-smith-frozen-trajectories-openai
SWE-Smith Frozen Trajectories — OpenAI Wire Format
This dataset is the OpenAI chat-completions wire-format release of
reflectio/swe-smith-frozen-trajectories,
derived from the tool split of
SWE-bench/SWE-smith-trajectories.
It is a serving-performance workload for realistic multi-turn coding-agent
histories. It can be used to measure request throughput, input/output token
throughput, TTFT, TPOT, streaming behavior, and prefix-cache reuse. It is not
a coding-correctness… See the full description on the dataset page: https://huggingface.co/datasets/reflectio/swe-smith-frozen-trajectories-openai.mscoco_train_2014_openai_clip-vit-base-patch32_image_caption_retrieval_pairs_2022-09-01openai-anthropic-price-index
OpenAI and Anthropic API Price Index — August 2026
A versioned research release containing normalized OpenAI and Anthropic API price records, source identifiers, methodology, machine-readable metadata, and a reusable provider-level chart bundle.
Dataset structure
The 212 normalized price records in the focused nine-offering OpenAI and Anthropic refresh.
Variables: provider_id, model_id, price_component, native_amount, native_unit, normalized_usd_per_1m_tokens… See the full description on the dataset page: https://huggingface.co/datasets/CurtisPyke/openai-anthropic-price-index.openai-hf-incident-recovered-tables
Recovered tables from the METR OpenAI and Hugging Face incident figures
This is an unofficial third party dataset. METR did not produce it, review it or endorse it.
METR's report on the OpenAI and Hugging Face incident contains two interactive charts. Those
charts load their numbers from JavaScript files on metr.org. This dataset holds those numbers
as CSV tables, with the source URLs, the SHA-256 of each source file, and a script that
downloads the sources again and checks the… See the full description on the dataset page: https://huggingface.co/datasets/LauraGomezjurado/openai-hf-incident-recovered-tables.flickr30k_openai_clip-vit-base-patch32_image_image_retrieval_pairs_2022-09-14mscoco_train_2014_openai_clip-vit-base-patch32_image_image_retrieval_pairs_2022-09-15openai-agent-traces4chanpol-openaimod
Dataset Card for "kjj0/4chanpol-openaimod"
Warning: offensive content.
This dataset contains 114M unique posts made between June 2016 and November 2019.
This is a variant of the dataset provided by Raiders of the Lost Kek: 3.5 Years of Augmented 4chan Posts from the Politically Incorrect Board.
We have deduplicated posts and stripped metadata to create an easily accessible collection of unique texts.
We have also provided OpenAI moderation scores. A variant without these scores can… See the full description on the dataset page: https://huggingface.co/datasets/kjj0/4chanpol-openaimod.openai_records
tags:
- observers
tags:
- observers
tags:
- observers
tags:
observers
Dataset Card for Dataset Name
Dataset Details
Dataset Description
Curated by: [More Information Needed]
Funded by [optional]: [More Information Needed]
Shared by [optional]: [More Information Needed]
Language(s) (NLP): [More Information Needed]
License: [More Information Needed]
Dataset Sources… See the full description on the dataset page: https://huggingface.co/datasets/codeslord/openai_records.ah_openai_st_dialog
Dataset Card for "ah_openai_st_dialog"
More Information needed
philosophai-openai-gpt-4o
Dataset Card for "philosophai-openai-gpt-4o"
More Information needed
evalap-comparing-openweight-with-openaigpt-oss-120b-113
Comparing openweight with openai/gpt-oss-120b (ID: 113)
Comparing openweight Albert-API with specific judge openai/gpt-oss-120b
Overview
This dataset contains 24 experiments
from the EvalAP evaluation platform.
Datasets: Assistant IA - QA, MFS_questions_v01
Models evaluated: openweight-large, openweight-medium, openweight-small
Metrics: energy_consumption, generation_time, gwp_consumption, judge_notator, judge_precision, nb_tokens_completion, nb_tokens_prompt… See the full description on the dataset page: https://huggingface.co/datasets/AgentPublic/evalap-comparing-openweight-with-openaigpt-oss-120b-113.ah_openai_qt_dialog
Dataset Card for "ah_openai_qt_dialog"
More Information needed
ah_openai_dialog
Dataset Card for "ah_openai_dialog"
More Information needed
openai-community-posts
OpenAI Community Posts
This dataset is curated from the posts of the OpenAI Community Forum (https://community.openai.com).
Dataset Details
Dataset Description
The OpenAI Community Posts dataset comprises discussions, posts, and metadata from the OpenAI Community Forum.
It includes details such as discussion titles, tags, views, reply counts, post content, sentiment scores, vector embeddings for content analysis, and identifiers linking posts to… See the full description on the dataset page: https://huggingface.co/datasets/julep-ai/openai-community-posts.openai-synthetic-data-skill-twin-classification_v3openai-synthetic-data-skill-twin-classification_v5