hotchpotch/bekko-embedding-v1-hard-negatives

hotchpotch/bekko-embedding-hard-negatives

hotchpotch/bekko-embedding-hard-negatives is a private collection of hard-negative training subsets for information retrieval and embedding-model training. Each row contains one query, one positive text, and 15 mined hard negatives in a flat schema:

query, pos, neg_1, neg_2, ..., neg_15

The dataset is intended for training dense retrievers, sparse/dense hybrid retrievers, embedding models, and rerankers that benefit from explicit challenging negatives. The collection currently covers English web text, biomedical abstracts, arXiv papers, code-search pairs, English Wikipedia paragraph retrieval, multilingual Wikipedia long-document retrieval, and several Sentence Transformers query-pair datasets.

The dataset card documents the durable upstream datasets and construction methods for the current hard-negative subsets. Some intermediate hard-negative datasets used during assembly may be removed; use this dataset and the upstream source references below as the stable references.

Loading

from datasets import load_dataset

ds = load_dataset(
    "hotchpotch/bekko-embedding-hard-negatives",
    "natural_questions",
    split="train",
)
print(ds[0].keys())

For multilingual Wikipedia long-document hard negatives:

from datasets import load_dataset

ds = load_dataset(
    "hotchpotch/bekko-embedding-hard-negatives",
    "wikipedia_hard_negatives_long_docs_ja",
    split="train",
)

Dataset Structure

All subsets expose the same columns.

The exported rows do not include document IDs, scores, rank positions, query-generation metadata, source labels, or mining diagnostics. If you need provenance-aware training or evaluation, keep a separate local mapping to the original source datasets.

Subsets

Total rows: 759,587.

ccnews is intentionally not included in this bundle because manual review found it less suitable than the retained hard-negative subsets.

Construction

Hybrid hard negatives

The fineweb, pubmed, arxiv, codesearch, natural_questions, agnews, trivia_qa, gooaq, all_nli, coco_captions, and wikipedia_hard_negatives_english subsets were mined with hybrid retrieval over Qdrant. The index combined sparse BM25 retrieval with dense vectors from an internal embedding model.

Each exported row contains 15 negatives. Rank ranges below are inclusive integer ranges.

The deeper top-k policy for gooaq, all_nli, and coco_captions was chosen after manual review found shallow negatives too easy, too repetitive, or too close to the positive.

Wikipedia multilingual long-document hard negatives

The multilingual long-document subsets are derived from Wikipedia long documents. They follow the same broad idea as `Shitao/MLDR`: a paragraph or evidence span inside a long document is used to generate a retrieval-style query, and the whole long document is the positive target. MLDR's dataset card states that it uses GPT-3.5-generated questions; these subsets instead use DeepSeek-generated queries over Wikipedia-derived source text.

The long-document hard negatives were mined per language with Qdrant sparse BM25 only. The BM25 index used body-only document text with the title prefix removed before indexing and in the final exported text.

Rows that could not provide all 15 filtered negatives were skipped.

Mining Diagnostics

Query Generation Sources

The non-Wikipedia 20k source datasets are based on upstream query-pair datasets whose dataset cards should be treated as the primary source of provenance:

• —FineWeb: simulated English search queries for FineWeb-Edu-derived web documents.
• —PubMed: simulated specialist biomedical search queries for PubMed title-and-abstract records.
• —arXiv: simulated specialist scientific search queries for arXiv title-and-abstract records.
• —Nomic codesearch: weakly supervised contrastive text-pair data from nomic-ai/nomic-embed-unsupervised-data, codesearch subset.
• —Sentence Transformers datasets: query-pair or sentence-pair data from the corresponding sentence-transformers/* upstream datasets.
• —English Wikipedia paragraphs: simulated English search queries over Wikipedia paragraph positives.

The multilingual Wikipedia long-document subsets use this query-generation prompt pattern:

You generate one query for training an information retrieval model for long-document retrieval.

The input is a paragraph selected from a long document. Write one useful search query whose answer is fully supported by that paragraph and whose wording helps retrieve the source document.

Requirements:
- Prefer the central entity, event, mechanism, relation, or claim of the paragraph over a minor side fact.
- The query must be specific enough to retrieve the source document, not a broad question that could match many unrelated documents.
- The query must include at least one paragraph-specific anchor, such as a named entity, work title, place, date, number, technical term, law, event, or organization.
- The query must ask about exactly one topic.
- The query must contain only one question.
- The query must be answerable using only the provided paragraph.
- Do not write a yes/no question; ask for a concrete entity, date, number, term, relationship, function, role, value, action, or explicitly stated explanation.
- Before writing the final query, choose an answer target from the paragraph. If that target looks malformed, mistranslated, noisy, semantically inconsistent, peripheral, or dependent on missing notation, reject it and choose a clearer central target.
- Preserve the exact spelling of names, titles, organizations, technical terms, and quoted terms from the paragraph.
- Prefer asking about a stated function, role, value, action, or relation over asking a broad definition such as "What is X?".
- When asking for a number, date, percentage, or count, make the measured subject explicit in the query.
- Do not ask for causes, effects, implications, background, proof, or interpretation unless the paragraph explicitly states them.
- If the paragraph contains broken formulas, missing symbols, placeholders, or unclear notation, do not ask about the missing notation; ask about a clearly stated named entity, date, number, or term instead.
- Do not make an article, source, paragraph, passage, or text the subject of the question.
- Do not mention "the text", "the paragraph", "the passage", or similar source references.
- Do not use pronouns such as "it", "this", "that", "these", or "those".
- Make the subject of the query explicit enough that the query remains meaningful outside the paragraph.
- Keep the query concise, grammatical, and natural.
- End the query with a question mark or the standard question punctuation for the output language.
- Output only the query. Do not include explanations, labels, quotes, or additional content.

Write the query in "{language name}".

Intended Use

This dataset is suitable for:

• —training dense retrieval and embedding models with explicit hard negatives;
• —training rerankers or cross-encoders that consume query, pos, and multiple negatives;
• —stress-testing retrieval models on lexically or semantically close negatives;
• —mixing domain-specific hard-negative data into a broader retrieval training recipe;
• —multilingual long-document retrieval experiments using Wikipedia-derived positives and BM25-only hard negatives.

Quality Notes

Manual review of the recently added Sentence Transformers hard-negative subsets suggests the following practical grouping:

• —Strong general-purpose additions: natural_questions, agnews, trivia_qa, and gooaq.
• —More specialized or lower-weight additions: all_nli and coco_captions.

all_nli contains short entailment-style pairs. Some mined negatives are semantically close to the positive, which can be useful for fine-grained semantic training but risky if treated as ordinary retrieval negatives. coco_captions contains caption-like texts where many examples describe visually similar scenes; this makes the negatives hard, but it can also create near-positive ambiguity. Use these subsets with lower sampling weight if your training objective assumes clearly non-relevant negatives.

License and Attribution

This aggregate follows the licenses and terms of the upstream datasets used to build each subset. The Hugging Face dataset metadata is preserved in this card; users should also check each upstream dataset card before redistribution or commercial use. Wikipedia-derived subsets should be treated as Wikipedia-derived content and are subject to Wikipedia's attribution and share-alike requirements, including CC BY-SA-style obligations. PubMed, arXiv, FineWeb, Nomic, and Sentence Transformers source subsets may have different license or usage constraints depending on their upstream data.