novelty
Datasets
All datasets matching “novelty”novelty-bench
NoveltyBench
Prompts and evaluation results for
NoveltyBench, which measures how many of a
model's ten sampled responses to a prompt are meaningfully different, and how
good those are. Code: novelty-bench/novelty-bench.
Prompts
data/curated-*.parquet (100 prompts written for the benchmark) and
data/wildchat-*.parquet (1,000 prompts from real ChatGPT conversations), with
fields id and prompt. These load as the curated and wildchat splits of
the default config.… See the full description on the dataset page: https://huggingface.co/datasets/yimingzhang/novelty-bench.autoresearch-novelty-bench
Autoresearch Novelty Bench
A benchmark for testing whether an autonomous AI research agent proposes
novel, mechanism-distinct hypotheses that anticipate breakthroughs
later found by other researchers.
By Evo. Built on Prime Intellect's
autonomous-speedrunning archive
— two AI agents (Claude Code and Codex) competing on modded-nanogpt's
optimization speedrun.
What's in this dataset
table
rows
description
experiments.parquet
10,380
One row per training run —… See the full description on the dataset page: https://huggingface.co/datasets/evo-hq/autoresearch-novelty-bench.novelty-rank-with-similaritiespg19-semantic-novelty
PG19 Semantic Novelty Dataset
Paragraph-by-paragraph semantic novelty curves for 28,535 books from the PG19 corpus (Project Gutenberg, pre-1920 English literature).
What is Semantic Novelty?
For each paragraph in a book, we compute:
novelty(p) = 1 - cosine_similarity(embedding(p), running_centroid)
where embedding() uses SBERT all-mpnet-base-v2 (768-dimensional) and running_centroid is the mean of all preceding paragraph embeddings. This measures how much new information… See the full description on the dataset page: https://huggingface.co/datasets/wfzimmerman/pg19-semantic-novelty.novelty-search-bench
PatSnap Novelty Search Bench
A Bench for evaluating patent novelty search (prior art search) systems. Each sample contains a query patent publication number (PN) along with ground truth X-type (novelty-destroying) prior art references identified by patent examiners.
The Bench is deliberately designed as a 50% public release of an internal full evaluation set that combines two complementary sample types — cross-jurisdiction family-expanded samples and single-jurisdiction… See the full description on the dataset page: https://huggingface.co/datasets/PatSnap/novelty-search-bench.novelty-bench-llm-judge