negative
Datasets
All datasets matching “negative”mmarco-hard-negatives-reranker-filtered
mMARCO Reranker-Filtered Hard Negatives (Multilingual)
Overview
This dataset is built from mMARCO (multilingual MS MARCO) triplets for each language subset. For each (query, positive), hard negatives are bundled and then filtered using cross-encoder re-scoring. The goal is to remove negatives that are too strong or incorrect for training. The same procedure is applied to all language subsets.
The dataset is published as mmarco-hard-negatives-reranker-filtered with… See the full description on the dataset page: https://huggingface.co/datasets/hotchpotch/mmarco-hard-negatives-reranker-filtered.rag_multilingual_training_negatives
How this dataset was made
We trained on chunks sourced from the documents in MADLAD-400 dataset that had been evaluated to contain a higher amount of educational information according to a state-of-the-art LLM.
We took chunks of size 250 tokens, 500 tokens, and 1000 tokens randomly for each document.
We then used these chunks to generate questions and answers based on this text using a state-of-the-art LLM.
Finally, we selected negatives for each chunk using the similarity from the… See the full description on the dataset page: https://huggingface.co/datasets/lightblue/rag_multilingual_training_negatives.bekko-embedding-v1-hard-negatives
hotchpotch/bekko-embedding-hard-negatives
hotchpotch/bekko-embedding-hard-negatives is a private collection of hard-negative training subsets for information retrieval and embedding-model training. Each row contains one query, one positive text, and 15 mined hard negatives in a flat schema:
query, pos, neg_1, neg_2, ..., neg_15
The dataset is intended for training dense retrievers, sparse/dense hybrid retrievers, embedding models, and rerankers that benefit from explicit… See the full description on the dataset page: https://huggingface.co/datasets/hotchpotch/bekko-embedding-v1-hard-negatives.finqa-hard-negatives
FinQA hard negatives
Near-duplicate hard negatives for training financial-document retrievers, built
from the train split of FinQA
(Chen et al., arXiv:2109.00122).
Each row is one question, its gold evidence unit(s), and the other evidence
units from the same filing page as hard negatives — same company, same
fiscal year, same boilerplate. The distinction a model must learn is which
sentence or table row on the page answers the question, not which company or
topic is relevant.… See the full description on the dataset page: https://huggingface.co/datasets/bowang0911/finqa-hard-negatives.msmarco-hard-negatives
MS MARCO Passages Hard Negatives
[!NOTE]
This repository contains raw datasets, all of which have also been formatted for easy training in the MS MARCO Mined Triplets collection. We recommend looking there first.
MS MARCO is a large scale information retrieval corpus that was created based on real user search queries using Bing search engine.
This dataset repository contains files that are helpful to train bi-encoder models e.g. using sentence-transformers.
Training Code… See the full description on the dataset page: https://huggingface.co/datasets/sentence-transformers/msmarco-hard-negatives.codeedit_hard_negative_datasets_kd