datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
gooaq_mt_german_5_hard_negatives
GooAQ (Google Answers to Google Questions) question-answer pairs in German with 5 mined hard negatives.
About
This dataset is a collection of ~2M question-answer-negative triplets and question-answer-negative_1...-negative_5 tuples from the machine translated version of MarcGrumpyOlejak/gooaq_mt_german. The full original Gooaq dataset in english only: (link to original dataset). This dataset can be used directly with Sentence Transformers to train embedding models.… See the full description on the dataset page: https://huggingface.co/datasets/MarcGrumpyOlejak/gooaq_mt_german_5_hard_negatives.gooaq_mt_german_0_hard_negatives
Remaining GooAQ (Google Answers to Google Questions) question-answer pairs in German without hard negatives.
About
This dataset contains the remaining 600K of lines of german machine translated texts of the mined hard negatives ~2M question-answer-negative triplets and question-answer-negative_1...-negative_5 tuples gooaq_mt_german_5_hard_negatives. The full original Gooaq dataset in english only: (link to original dataset). This dataset can be used directly with Sentence… See the full description on the dataset page: https://huggingface.co/datasets/MarcGrumpyOlejak/gooaq_mt_german_0_hard_negatives.multihop_qa_sft-hard-negatives
multihop_qa_sft — hard negative IDs
Hard negatives mined for ragrawal36/multihop_qa_sft
(train split). IDs only — they index the document corpus at
mihir-1999/multihop_qa_sft-doc-corpus.
Schema
column
type
meaning
row_id
int32
source row index in ragrawal36/multihop_qa_sft train
pos_doc_ids
list[int32]
the row's own supporting docs (positives)
neg_doc_ids
list[int32]
200 mined hard negatives
pos_doc_ids and neg_doc_ids are disjoint by… See the full description on the dataset page: https://huggingface.co/datasets/mihir-1999/multihop_qa_sft-hard-negatives.nepali-query-passage-hard-negatives-10kmmarco-more-hard-negatives
mMarco with more hard negatives
At least 5 hard negatives per each pair (query, answer) on training set
On eval set 30 hard negatives for each pair (query, answer).
Here it is a Dataset mined to finetune a reranker.
Feel free to resplit the dataset.
Who else have a lot of gpu's and a cpu with over 128 cores take a look on this :)
Model Card Authors
André Ribeiro @andreribeiro87
Rúben Garrido @RGarrido03