datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
piqapiqa
Dataset Card for "Physical Interaction: Question Answering"
Dataset Summary
To apply eyeshadow without a brush, should I use a cotton swab or a toothpick?
Questions requiring this kind of physical commonsense pose a challenge to state-of-the-art
natural language understanding systems. The PIQA dataset introduces the task of physical commonsense reasoning
and a corresponding benchmark dataset Physical Interaction: Question Answering or PIQA.
Physical commonsense knowledge… See the full description on the dataset page: https://huggingface.co/datasets/lighteval/piqa.global-piqa-nonparallel
Global PIQA Non-Parallel
Global PIQA is a participatory commonsense reasoning benchmark for over 100 languages, constructed by hand by over 350 researchers from over 65 countries around the world.
The non-parallel split covers 136 language varieties, covering five continents, 18 language families, and 24 writing systems.
In this non-parallel split, over 50% of examples reference local foods, customs, traditions, or other culturally-specific elements.
Details are in our preprint:… See the full description on the dataset page: https://huggingface.co/datasets/mrlbenchmarks/global-piqa-nonparallel.piqaglobal-piqa-parallel
Global PIQA Parallel
Global PIQA is a participatory commonsense reasoning benchmark for over 100 languages, constructed by hand by over 350 researchers from over 65 countries around the world.
The parallel split is a multi-parallel dataset for 131 language varieties, covering five continents, 16 language families, and 23 writing systems.
In this parallel split, each example was machine-translated from English, then manually corrected by a native speaker of the target language.… See the full description on the dataset page: https://huggingface.co/datasets/mrlbenchmarks/global-piqa-parallel.piqapiqaThe PIQA dataset without needing to run remote code, so it is compatible with datasets >= 4.0.0.
piqa
Dataset Card for "piqa"
More Information needed
Note: This dataset was utilized for the evaluation of probability-based prompt selection techniques in the paper 'Improving Probability-based Prompt Selection Through Unified Evaluation and Analysis'. It differs from the actual benchmark dataset.
llama2_7b_chat-piqa-resultspiqa-multilingualpiqa-train-10ktestset_piqatask081_piqa_wrong_answer_generation
Dataset Card for Natural Instructions (https://github.com/allenai/natural-instructions) Task: task081_piqa_wrong_answer_generation
Additional Information
Citation Information
The following paper introduces the corpus in detail. If you use the corpus in published work, please cite it:
@misc{wang2022supernaturalinstructionsgeneralizationdeclarativeinstructions,
title={Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks}… See the full description on the dataset page: https://huggingface.co/datasets/Lots-of-LoRAs/task081_piqa_wrong_answer_generation.task080_piqa_answer_generation
Dataset Card for Natural Instructions (https://github.com/allenai/natural-instructions) Task: task080_piqa_answer_generation
Additional Information
Citation Information
The following paper introduces the corpus in detail. If you use the corpus in published work, please cite it:
@misc{wang2022supernaturalinstructionsgeneralizationdeclarativeinstructions,
title={Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks}… See the full description on the dataset page: https://huggingface.co/datasets/Lots-of-LoRAs/task080_piqa_answer_generation.piqa_preop
Dataset Card for "piqa_preop"
More Information needed
AraDiCE-PIQA
AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMs
Overview
The AraDiCE dataset is designed to evaluate dialectal and cultural capabilities in large language models (LLMs). The dataset consists of post-edited versions of various benchmark datasets, curated for validation in cultural and dialectal contexts relevant to Arabic. In this repository we show the PIQA split of the data
Evaluation
We have used lm-harness eval framework to for the… See the full description on the dataset page: https://huggingface.co/datasets/QCRI/AraDiCE-PIQA.piqapiqa_ca
Dataset Card for piqa_ca
piqa_ca is a multiple choice question answering dataset in Catalan that has been professionally translated from the PIQA validation set in English.
Dataset Details
Dataset Description
piqa_ca (Physical Interaction Question Answering - Catalan) is designed to evaluate physical commonsense reasoning using question-answer triplets based on everyday situations. It includes 1838 instances in the validation split. Each instance contains… See the full description on the dataset page: https://huggingface.co/datasets/projecte-aina/piqa_ca.PIQA
PIQA
An MTEB dataset
Massive Text Embedding Benchmark
Measuring the ability to retrieve the groundtruth answers to reasoning task queries on PIQA.
Task category
t2t
Domains
Encyclopaedic, Written
Reference
https://arxiv.org/abs/1911.11641
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_task("PIQA")
evaluator = mteb.MTEB([task])
model = mteb.get_model(YOUR_MODEL)… See the full description on the dataset page: https://huggingface.co/datasets/mteb/PIQA.PIQA-eu
Dataset Card for PIQA-eu
Point of Contact: hitz@ehu.eus
Dataset Description
Dataset Summary
PIQA-eu is the professional translation to Basque of the PIQA's
(Bisk et al., 2020) validation partition.
PIQA is a commonsense QA benchmark for naive physics reasoning focusing on how we interact with everyday
objects in everyday situations.
Languages
eu-ES
Dataset Structure
Data Instances
PIQA-eu examples look like this:
{… See the full description on the dataset page: https://huggingface.co/datasets/HiTZ/PIQA-eu.piqaKo-PIQA
Ko-PIQA: Korean Physical Commonsense Reasoning Dataset
📖 Dataset Overview
Ko-PIQA is a Korean Physical Commonsense Reasoning dataset designed to complement English-centric benchmarks like PIQA and to include culturally-grounded physical reasoning questions.
Total items: 441
Culturally-grounded items: 87 (19.7%)(e.g., kimchi storage, hanbok care, ondol heating)
Format: PIQA-style binary choice (solution0 / solution1)
Goal: Evaluate Korean LLM physical reasoning… See the full description on the dataset page: https://huggingface.co/datasets/HAERAE-HUB/Ko-PIQA.piqapiqa_indic
Dataset Card for Dataset Name
Dataset Summary (Taken from Piqa)
To apply eyeshadow without a brush, should I use a cotton swab or a toothpick? Questions requiring this kind of physical commonsense pose a challenge to state-of-the-art natural language understanding systems. The PIQA dataset introduces the task of physical commonsense reasoning and a corresponding benchmark dataset Physical Interaction: Question Answering or PIQA.
Physical commonsense knowledge is a major challenge on… See the full description on the dataset page: https://huggingface.co/datasets/iitrsamrat/piqa_indic.kor_piqa
Dataset Card for "kor_piqa"
More Information needed
Source Data Citation Information
@inproceedings{Bisk2020,
author = {Yonatan Bisk and Rowan Zellers and
Ronan Le Bras and Jianfeng Gao
and Yejin Choi},
title = {PIQA: Reasoning about Physical Commonsense in
Natural Language},
booktitle = {Thirty-Fourth AAAI Conference on
Artificial Intelligence},
year = {2020},
}
PIQApiqa-bn
Dataset Summary
This is the translated version of the PIQA LLM evaluation dataset. The dataset was translated using a new method called Expressive Semantic Translation (EST), which combines Google Translation with LLM-based rewriting. PIQA introduces the task of physical commonsense reasoning and provides a corresponding benchmark for understanding physical interactions in everyday situations. It focuses on atypical solutions to practical problems, inspired by instructional guides… See the full description on the dataset page: https://huggingface.co/datasets/hishab/piqa-bn.piqa_italian
PIQA - Italian (IT)
This dataset is an Italian translation of PIQA. PIQA stands for Physical Interaction Question Answering, a dataset of questions about common scenarios that require an understanding of the physical world.
Dataset Details
The dataset consists of questions about common scenarios that require an understanding of the physical world. Each question is associated with a correct answer and a distractor. The task is to predict the correct answer to the… See the full description on the dataset page: https://huggingface.co/datasets/sapienzanlp/piqa_italian.piqa_ru
Dataset Card for "piqa_ru"
This is translated version of piqa dataset into Russian.
piqa_with_dev_hpo