datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
lfqalfqa_support_docsSupport documents for building https://huggingface.co/vblagoje/bart_lfqa model
lfqa_idlfqa-preprocessed-itlfqa_preprocessed
Dataset Card for "lfqa_preprocessed"
Dataset Summary
This is a simplified version of vblagoje's lfqa_support_docs and lfqa datasets.
It was generated by me to have a more straight forward way to train Seq2Seq models on context based long form question answering tasks.
Dataset Structure
Data Instances
An example of 'train' looks as follows.
{
"question": "what's the difference between a forest and a wood?",
"answer": "They're used… See the full description on the dataset page: https://huggingface.co/datasets/LLukas22/lfqa_preprocessed.eli5-lfqa
ELI5: Long Form Question Answering
https://arxiv.org/pdf/1907.09190
Items (train): 272_634
Items (val): 1_507
Downloaded from:
https://www.kaggle.com/datasets/trandaiphu/eli5-dataset
Tokens ctxs length distribution summary:
min: 152
mean: 3745.35
median (p50): 3736.00
p90: 4037.00
p95: 4152.00
p99: 4432.00
max: 6530
Tokens answers length distribution summary:
rows: 272_634
min: 8
mean: 401.31
median (p50): 217.00
p90: 820.70
p95: 1230.00
p99:… See the full description on the dataset page: https://huggingface.co/datasets/aitetic/eli5-lfqa.intent-aware-lfqa-intent-impliciteli5-lfqa-combined
eli5-lfqa-combined
Postprocessed from eli5-lfqa: (ctxs, question+answers[])
Assembled by: dataset_assembler.py
Size: 1.1B
lfqa-textbuggerintent-aware-lfqa-multiviewintent-aware-lfqa-baselineNAITS_LFQA_with_supportsintent-aware-lfqa-intent-explicitLFQAKnowledgeBaseLFQA-HP-1M-Samplelfqa-idlfqa_test_with_supports_v1NAITS_LFQA_with_supports_v2
NAITS_LFQA_with_supports_v2
Dataset Description
NAITS_LFQA_with_supports_v2 is a bilingual Arabic-English dataset designed for Long-Form Question Answering (LFQA) and Retrieval-Augmented Generation (RAG) research.
The dataset contains 102 manually curated samples. Each sample consists of:
A question in Arabic and English.
A long-form answer in Arabic and English.
Supporting passages in Arabic and English from which the answer can be derived.
A reference field… See the full description on the dataset page: https://huggingface.co/datasets/fahdsoliman/NAITS_LFQA_with_supports_v2.lfqa_datasetlfqa_test_with_supports_v2lfqa-deepwordbug