datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
perturbed-knights-and-knaves
📘 perturbed-knights-and-knaves Dataset [Project Page]
The perturbed-knights-and-knaves dataset evaluates the consistency of LLMs' logical reasoning ability under various perturbations.
🚀🚀 Check out the clean version of the dataset at [knights-and-knaves].
Loading the dataset
To load the dataset:
from datasets import load_dataset
data_subject = datasets.load_dataset('K-and-K/perturbed-knights-and-knaves', data_files="{subset}/{perturbation}/{subject}.jsonl")… See the full description on the dataset page: https://huggingface.co/datasets/K-and-K/perturbed-knights-and-knaves.Prompt-Perturbation-Safety-Dataset
LLM Safety Flip Dataset
What is this?
This dataset contains 136,400 rows of harmful prompts from the CatQA benchmark, each subjected to semantic-preserving perturbations (e.g., typos, insertions, paraphrasing). Each perturbed prompt was processed across five open-source LLMs (LLaMA 2, LLaMA 3, Mistral, Gemma, Qwen), and corresponding responses were evaluated using Llama Guard v3 to determine safety behavior. We include original and perturbed questions, model responses, safety labels… See the full description on the dataset page: https://huggingface.co/datasets/Ztrimus/Prompt-Perturbation-Safety-Dataset.math_perturbed_200
MATH Perturbed (200 Problems)
A dataset of 200 MATH benchmark problems where numerical values have been changed, resulting in different correct answers. Created to study test set contamination and whether models have memorized specific answers versus learned mathematical reasoning.
Dataset Description
This dataset contains 200 problems from the MATH benchmark where numerical values have been modified using Claude. Each problem maintains the same structure and problem type… See the full description on the dataset page: https://huggingface.co/datasets/stellaathena/math_perturbed_200.error-detection-positives_perturbed
error-detection-positives_perturbed
This dataset is part of the PARC (Premise-Annotated Reasoning Collection) and contains mathematical reasoning problems with error annotations. This dataset combines positives_perturbed samples from multiple domains.
Domain Breakdown
gsm8k: 48 samples
math: 42 samples
metamathqa: 72 samples
orca_math: 85 samples
Features
Each example contains:
data_source: The domain/source of the problem (gsm8k, math, metamathqa… See the full description on the dataset page: https://huggingface.co/datasets/PARC-DATASETS/error-detection-positives_perturbed.