datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
squad_v2
Dataset Card for SQuAD 2.0
Dataset Summary
Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
SQuAD 2.0 combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers… See the full description on the dataset page: https://huggingface.co/datasets/rajpurkar/squad_v2.squad_kor_v2KorQuAD 2.0 is a Korean question and answering dataset consisting of a total of 100,000+ pairs. There are three major differences from KorQuAD 1.0, which is the standard Korean Q & A data. The first is that a given document is a whole Wikipedia page, not just one or two paragraphs. Second, because the document also contains tables and lists, it is necessary to understand the document structured with HTML tags. Finally, the answer can be a long text covering not only word or phrase units, but paragraphs, tables, and lists. As a baseline model, BERT Multilingual is used, released by Google as an open source. It shows 46.0% F1 score, a very low score compared to 85.7% of the human F1 score. It indicates that this data is a challenging task. Additionally, we increased the performance by no-answer data augmentation. Through the distribution of this data, we intend to extend the limit of MRC that was limited to plain text to real world tasks of various lengths and formats.squad_es_v2automatic translation of the Stanford Question Answering Dataset (SQuAD) v2 into Spanishlong_squad_v2
Dataset Card for long_squad_v2
long_squad_v2 is a long-context question answering dataset based on the SQuAD v2 format. It was constructed by concatenating multiple SQuAD v2 contexts to significantly increase the average document length, enabling training and evaluation of models on long-range understanding and sparse answer retrieval tasks.
Dataset Details
Uses
To load the dataset using the 🤗 Datasets library:
from datasets import load_dataset
dataset =… See the full description on the dataset page: https://huggingface.co/datasets/huutuan/long_squad_v2.squad-augmented-v2squad-nl-v2.0
SQuAD-NL v2.0 [translated SQuAD / XQuAD]
SQuAD-NL v2.0 is a translation of The Stanford Question Answering Dataset (SQuAD) v2.0.
Since the original English SQuAD test data is not public, we reserve the same documents that were used for XQuAD for testing purposes. These documents are sampled from the original dev data split. The English data is automatically translated using Google Translate (February 2023) and the test data is manually post-edited.
This version of SQuAD-NL also… See the full description on the dataset page: https://huggingface.co/datasets/GroNLP/squad-nl-v2.0.squad_v2_svSQuAD_v2_sv is a Swedish version of SQuAD2.0. Translation was done automatically by using Google Translate API but it is not so straightforward because;
1. the span which determines the start and the end of the answer in the context may vary after translation,
2. tne translated context may not contain the translated answer if we translate both independently.
More details on how to handle these will be provided in another blog post.chinese-squadv2English | 中文
Dataset Card for Chinese SQuAD 2.0 (revised, bilingual)
Dataset Description
This is a revised and extended version of the Chinese translation of SQuAD 2.0,
originally machine-translated by
ChineseSquad. Like SQuAD 2.0 it
contains both answerable and unanswerable questions and is designed for Chinese
extractive reading comprehension / question answering.
Compared with the previous release of chinese-squadv2, this version:
Adds the original English… See the full description on the dataset page: https://huggingface.co/datasets/real-jiakai/chinese-squadv2.squad_v2.0squad_v2_dutch
Dataset Card for "squad_v2_dutch"
Deprecated: This translation is not recommended. 12% of the translated answers do not appear verbatim in the contexts. Use NetherlandsForensicInstitute/squad-nl-v2.0 instead.
Dataset Summary
The squad_v2_dutch dataset is a machine-translated version of the SQuAD v2 dataset from English to Dutch.
The SQuAD v2 dataset combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by… See the full description on the dataset page: https://huggingface.co/datasets/yhavinga/squad_v2_dutch.kor_squad_v2
Dataset Card for squad_v2
Licensing Information
The data is distributed under the CC BY SA 4.0 license.
Source Data Citation Information
@article{2016arXiv160605250R,
author = {{Rajpurkar}, Pranav and {Zhang}, Jian and {Lopyrev},
Konstantin and {Liang}, Percy},
title = "{SQuAD: 100,000+ Questions for Machine Comprehension of Text}",
journal = {arXiv e-prints},
year = 2016,
eid =… See the full description on the dataset page: https://huggingface.co/datasets/KETI-NLP/kor_squad_v2.squad_v2_ficombines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers
to look similar to answerable ones. To do well on SQuAD2.0, systems must not only answer questions when possible, but
also determine when no answer is supported by the paragraph and abstain from answering.finbenchv2-squad_v2-fi-mt
Dataset Summary
This is a Finnish SQuAD question answering dataset used in FIN-bench-v2: A Unified and Robust Benchmark Suite for Evaluating Finnish Large Language Models. It is a DeepL-based machine translation of the English SQuAD2.0 dataset which combines the 100,000 questions in
SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers to look similar to answerable ones.
To do well on SQuAD2.0, systems must not only answer questions when possible… See the full description on the dataset page: https://huggingface.co/datasets/TurkuNLP/finbenchv2-squad_v2-fi-mt.SQuAD_v2_fi
Dataset Card for "squad-v2-fi"
Dataset Summary
Machine translated and normalized Finnish version of the SQuAD-v2.0 dataset. Details about the translation and normalization processes can be found here.
Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the… See the full description on the dataset page: https://huggingface.co/datasets/ilmariky/SQuAD_v2_fi.squad-nl-v2.0
SQuAD-NL v2.0 for Sentence Transformers
The SQuAD-NL v2.0 dataset (on Hugging Face: GroNLP/squad-nl-v2.0), modified for use in Sentence Transformers as a dataset of type "Pair with Similarity Score".
Score
We added an extra column score to the original dataset.
The value of score is 1.0 if the question has an answer in the context (no matter where), and 0.0 if there are no answers in the context.
The allows the evaluation of embedding models that aim to pair queries… See the full description on the dataset page: https://huggingface.co/datasets/NetherlandsForensicInstitute/squad-nl-v2.0.squad_v2_fi_hfThis dataset is part of the FIN-bench-v2: A Unified and Robust Benchmark Suite for Evaluating Finnish Large Language Models paper.
Project Page: https://huggingface.co/TurkuNLP
Code: https://github.com/LumiOpen/lm-evaluation-harness
squad_v2_french_translated_fr_prompt_qa
squad_v2_french_translated_fr_prompt_qa
Summary
squad_v2_french_translated_fr_prompt_qa is a subset of the Dataset of French Prompts (DFP).It contains 3,320,898 rows that can be used for a question-answering task.The original data (without prompts) comes from the dataset pragnakalp/squad_v2_french_translated and was augmented by questions in SQUAD 2.0 format in the FrenchQA dataset.
A list of prompts (see below) was then applied in order to build the input and target… See the full description on the dataset page: https://huggingface.co/datasets/CATIE-AQ/squad_v2_french_translated_fr_prompt_qa.squad_v2
Dataset Card for SQuAD 2.0
Dataset Summary
Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
SQuAD 2.0 combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers… See the full description on the dataset page: https://huggingface.co/datasets/Divyanshu67/squad_v2.squad_v2
Dataset Card for SQuAD 2.0
Dataset Summary
Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
SQuAD 2.0 combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers… See the full description on the dataset page: https://huggingface.co/datasets/superyuhan/squad_v2.squad-v2-lance
SQuAD v2 (Lance Format)
A Lance-formatted version of SQuAD v2 — the Stanford Question Answering Dataset with both answerable and deliberately unanswerable questions over Wikipedia passages — with MiniLM question embeddings stored inline and ready for retrieval at hf://datasets/lance-format/squad-v2-lance/data.
Key features
Span-extraction QA over Wikipedia with 130k+ training questions and an is_impossible flag that cleanly separates answerable from unanswerable items.… See the full description on the dataset page: https://huggingface.co/datasets/lance-format/squad-v2-lance.clean_squad_v2
Clean SQuAD v2
This is a refined version of the SQuAD v2 dataset. It has been preprocessed to ensure higher data quality and usability for NLP tasks such as Question Answering.
Description
The Clean SQuAD v2 dataset was created by applying preprocessing steps to the original SQuAD v2 dataset, including:
Trimming whitespace: All leading and trailing spaces have been removed from the question field.
Minimum question length: Questions with fewer than 12 characters were… See the full description on the dataset page: https://huggingface.co/datasets/decodingchris/clean_squad_v2.squad_v2_ptcombines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers
to look similar to answerable ones. To do well on SQuAD2.0, systems must not only answer questions when possible, but
also determine when no answer is supported by the paragraph and abstain from answering.clean_squad_classic_v2
Clean SQuAD Classic v2
This is a refined version of the SQuAD v2 dataset. It has been preprocessed to ensure higher data quality and usability for NLP tasks such as Question Answering.
Description
The Clean SQuAD Classic v2 dataset was created by applying preprocessing steps to the original SQuAD v2 dataset, including:
Trimming whitespace: All leading and trailing spaces have been removed from the question field.
Minimum question length: Questions with fewer than 12… See the full description on the dataset page: https://huggingface.co/datasets/decodingchris/clean_squad_classic_v2.train-indo-squad-v2-test
Dataset Card for "train-indo-squad-v2-test"
More Information needed