datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
paws
Dataset Card for PAWS: Paraphrase Adversaries from Word Scrambling
Dataset Summary
PAWS: Paraphrase Adversaries from Word Scrambling
This dataset contains 108,463 human-labeled and 656k noisily labeled pairs that feature the importance of modeling structure, context, and word order information for the problem of paraphrase identification. The dataset has two subsets, one based on Wikipedia and the other one based on the Quora Question Pairs (QQP) dataset.
For further… See the full description on the dataset page: https://huggingface.co/datasets/google-research-datasets/paws.pa-warm-start-sft-heavy-25b-mix
geodesic-research/pa-warm-start-sft-heavy-25b-mix
Auto-generated by dataset-builder.
Each config below is a separate dataset produced from a versioned YAML build
config. Load with:
from datasets import load_dataset
ds = load_dataset("geodesic-research/pa-warm-start-sft-heavy-25b-mix", "<config_name>", revision="<commit-sha>")
Pin revision= to the specific commit SHA you want; without it, you get the
current HEAD of the dataset repo, which may change when the builder re-pushes.… See the full description on the dataset page: https://huggingface.co/datasets/geodesic-research/pa-warm-start-sft-heavy-25b-mix.paws-x
Dataset Card for PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification
Dataset Summary
This dataset contains 23,659 human translated PAWS evaluation pairs and
296,406 machine translated training pairs in six typologically distinct
languages: French, Spanish, German, Chinese, Japanese, and Korean. All
translated pairs are sourced from examples in
PAWS-Wiki.
For further details, see the accompanying paper:
PAWS-X: A Cross-lingual Adversarial Dataset for… See the full description on the dataset page: https://huggingface.co/datasets/google-research-datasets/paws-x.pa-warm-start-sft-xl-50b-mix
geodesic-research/pa-warm-start-sft-xl-50b-mix
Auto-generated by dataset-builder.
Each config below is a separate dataset produced from a versioned YAML build
config. Load with:
from datasets import load_dataset
ds = load_dataset("geodesic-research/pa-warm-start-sft-xl-50b-mix", "<config_name>", revision="<commit-sha>")
Pin revision= to the specific commit SHA you want; without it, you get the
current HEAD of the dataset repo, which may change when the builder re-pushes.… See the full description on the dataset page: https://huggingface.co/datasets/geodesic-research/pa-warm-start-sft-xl-50b-mix.pawsx_mtPAWS-X, a multilingual version of PAWS (Paraphrase Adversaries from Word Scrambling) for six languages.
This dataset contains 23,659 human translated PAWS evaluation pairs and 296,406 machine
translated training pairs in six typologically distinct languages: French, Spanish, German,
Chinese, Japanese, and Korean. English language is available by default. All translated
pairs are sourced from examples in PAWS-Wiki.
For further details, see the accompanying paper: PAWS-X: A Cross-lingual Adversarial Dataset
for Paraphrase Identification (https://arxiv.org/abs/1908.11828)
NOTE: There might be some missing or wrong labels in the dataset and we have replaced them with -1.PawsXPairClassification
PawsXPairClassification
An MTEB dataset
Massive Text Embedding Benchmark
{PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification
Task category
t2t
Domains
Web, Encyclopaedic, Written
Reference
https://arxiv.org/abs/1908.11828
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_task("PawsXPairClassification")
evaluator = mteb.MTEB([task])
model =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/PawsXPairClassification.pawn-stockfish-100m
PAWN Stockfish 100M
100,000,000 self-play chess games generated with Stockfish 18, each
annotated with per-position, per-legal-move evaluations — for chess
policy-learning and NNUE-distillation research.
Dataset Summary
100,000,000 machine-generated self-play chess games. Every position in
every game is annotated with an evaluation of every legal move, not just
the move played. The dataset was built as training data for
PAWN — a testbed for finetuning
and… See the full description on the dataset page: https://huggingface.co/datasets/thomas-schweich/pawn-stockfish-100m.pa-warm-start-sft-xl-smokePAWBench
PAWBench
A benchmark for distributional physical realism in generated videos
PAWBench asks whether a video model reproduces the range and frequency of
outcomes that a physical scene can produce. It evaluates repeated rollouts from
the same source image and action, rather than treating one plausible-looking
video as sufficient evidence of physical realism.
Paper ·
Code ·
Project website
At a glance
Component
Contents
Scenes
50 physical… See the full description on the dataset page: https://huggingface.co/datasets/Andrew613/PAWBench.pawnocchio-relabelled-20260713pa-warm-start-sft-medium-5b-mix
geodesic-research/pa-warm-start-sft-medium-5b-mix
Auto-generated by dataset-builder.
Each config below is a separate dataset produced from a versioned YAML build
config. Load with:
from datasets import load_dataset
ds = load_dataset("geodesic-research/pa-warm-start-sft-medium-5b-mix", "<config_name>", revision="<commit-sha>")
Pin revision= to the specific commit SHA you want; without it, you get the
current HEAD of the dataset repo, which may change when the builder re-pushes.… See the full description on the dataset page: https://huggingface.co/datasets/geodesic-research/pa-warm-start-sft-medium-5b-mix.pa-warm-start-sft-xl-calibrationtool-use-llama-format
Open Paws Tool Use Llama Format
This dataset is part of the Open Paws initiative to develop AI training data aligned with animal liberation and advocacy principles. Created to train AI systems that understand and promote animal welfare, rights, and liberation.
Dataset Details
Dataset Type: Tool Use Data
Format: JSONL (JSON Lines)
Languages: Multilingual (primarily English)
Focus: Animal advocacy and ethical reasoning
Organization: Open Paws
License: Apache 2.0… See the full description on the dataset page: https://huggingface.co/datasets/open-paws/tool-use-llama-format.visual-qa-llama-format
Open Paws Visual Qa Llama Format
This dataset is part of the Open Paws initiative to develop AI training data aligned with animal liberation and advocacy principles. Created to train AI systems that understand and promote animal welfare, rights, and liberation.
Dataset Details
Dataset Type: Multimodal Data
Format: JSONL (JSON Lines)
Languages: Multilingual (primarily English)
Focus: Animal advocacy and ethical reasoning
Organization: Open Paws
License: Apache 2.0… See the full description on the dataset page: https://huggingface.co/datasets/open-paws/visual-qa-llama-format.pa-warm-start-sft-heavy-25b-mix-longCUB
CUB (Image Retrieval)
This repository contains the raw images for the CUB dataset, formatted for use with Dino v3 retrieval applications.
Usage in Frontend
You can access these images directly via the Hugging Face resolve endpoint:
https://huggingface.co/datasets/<USERNAME>/CUB/resolve/main/<PATH_TO_FILE>
pa-warm-start-sft-xl-1b-smokeCars196
Cars196 (Image Retrieval)
This repository contains the raw images for the Cars196 dataset, formatted for use with Dino v3 retrieval applications.
Usage in Frontend
You can access these images directly via the Hugging Face resolve endpoint:
https://huggingface.co/datasets/<USERNAME>/Cars196/resolve/main/<PATH_TO_FILE>
pawn-lichess-full
PAWN Lichess Full
Rated Lichess games from Q1 2025 plus a January 2026 holdout, pre-tokenized in the PAWN v1.0.0 training format. Primarily intended finetuning the PAWN backbones on real human play. The dataset also keeps the raw SAN and UCI move strings, full Lichess metadata, and clock annotations, so it works as a general-purpose pre-parsed Lichess feed even outside the PAWN ecosystem — see Other uses below.
Splits
Split
Source
Games
Shards
train
January… See the full description on the dataset page: https://huggingface.co/datasets/thomas-schweich/pawn-lichess-full.PAWBench-Results
PAWBench Eval Package 20260708
This dataset package defines the clean HF repository layout and contains the
public metadata needed to run the PAWBench Full31 coverage and Table1 PAW-Cal
PAWEval rerun inputs. In the local no-copy handoff, videos are not duplicated
under this directory; they are uploaded into sharded videos/by_media_id/
subdirectories from the private upload plan after explicit human approval.
Layout
videos/by_media_id/<sha-prefix>/<media_id>.mp4:… See the full description on the dataset page: https://huggingface.co/datasets/Andrew613/PAWBench-Results.pa-warm-start-sft-light-1b-mix
geodesic-research/pa-warm-start-sft-light-1b-mix
Auto-generated by dataset-builder.
Each config below is a separate dataset produced from a versioned YAML build
config. Load with:
from datasets import load_dataset
ds = load_dataset("geodesic-research/pa-warm-start-sft-light-1b-mix", "<config_name>", revision="<commit-sha>")
Pin revision= to the specific commit SHA you want; without it, you get the
current HEAD of the dataset repo, which may change when the builder re-pushes.… See the full description on the dataset page: https://huggingface.co/datasets/geodesic-research/pa-warm-start-sft-light-1b-mix.paws-xinstagram-dbpa-warm-start-sft-xl-50b-mix-metagaming-filteredPAWSX
PAWSX
An MTEB dataset
Massive Text Embedding Benchmark
A Chinese dataset for textual relatedness
Task category
t2t
Domains
None
Reference
https://aclanthology.org/2021.emnlp-main.357
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["PAWSX"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)
evaluator.run(model)
To learn more about how to run models… See the full description on the dataset page: https://huggingface.co/datasets/mteb/PAWSX.proteingym-fm-benchmark
Protein Foundation Model Benchmark Results
Zero-shot fitness prediction results for protein foundation models evaluated on
the ProteinGym substitution benchmark (217 DMS
assays, ~2.7M variants).
Companion data for the paper: "From Sequence Encoders to Multimodal Systems:
A Critical Survey of Protein Foundation Models" (IEEE TCBB 2026).
Dataset configurations
The dataset viewer exposes two configurations, because the files carry two
different schemas that must not… See the full description on the dataset page: https://huggingface.co/datasets/PawanRamaMali/proteingym-fm-benchmark.pawsx_mt_triplet
PAWS-X Multilingual Triplet Dataset
This dataset contains PAWS-X (Paraphrase Adversaries from Word Scrambling) data organized by translation models for paraphrase detection and text similarity tasks.
Dataset Structure
Each sample contains the following fields:
id: Unique identifier for the text pair
text1: First sentence (originally sentence1)
text2: Second sentence (originally sentence2)
label: Binary label (1 for paraphrase, 0 for non-paraphrase)
model: Translation… See the full description on the dataset page: https://huggingface.co/datasets/patrickamadeus/pawsx_mt_triplet.mllm-shap
MLLM-SHAP experiment datasets
Curated test splits for studying Shapley-value explanations in multimodal large language models (text and audio inputs). Each configuration is a filtered, size-controlled subset built for reproducible benchmarking—not a full copy of the upstream corpora.
Configs follow the naming pattern {task}__{source} (for example single_sentence__voice_bench).
Quick load
Pin a dataset revision for reproducibility (replace REVISION with the commit hash… See the full description on the dataset page: https://huggingface.co/datasets/Pawlo77/mllm-shap.PAWSX
Dataset Card for "PAWSX"
More Information needed
move_pawn_forward_1This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "so100_follower",
"total_episodes": 47,
"total_frames": 15430,
"total_tasks": 1,
"total_videos": 94,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 30,
"splits": {
"train": "0:47"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/lumal/move_pawn_forward_1.