datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
mmarco-hard-negatives-reranker-filtered
mMARCO Reranker-Filtered Hard Negatives (Multilingual)
Overview
This dataset is built from mMARCO (multilingual MS MARCO) triplets for each language subset. For each (query, positive), hard negatives are bundled and then filtered using cross-encoder re-scoring. The goal is to remove negatives that are too strong or incorrect for training. The same procedure is applied to all language subsets.
The dataset is published as mmarco-hard-negatives-reranker-filtered with… See the full description on the dataset page: https://huggingface.co/datasets/hotchpotch/mmarco-hard-negatives-reranker-filtered.rag_multilingual_training_negatives
How this dataset was made
We trained on chunks sourced from the documents in MADLAD-400 dataset that had been evaluated to contain a higher amount of educational information according to a state-of-the-art LLM.
We took chunks of size 250 tokens, 500 tokens, and 1000 tokens randomly for each document.
We then used these chunks to generate questions and answers based on this text using a state-of-the-art LLM.
Finally, we selected negatives for each chunk using the similarity from the… See the full description on the dataset page: https://huggingface.co/datasets/lightblue/rag_multilingual_training_negatives.bekko-embedding-v1-hard-negatives
hotchpotch/bekko-embedding-hard-negatives
hotchpotch/bekko-embedding-hard-negatives is a private collection of hard-negative training subsets for information retrieval and embedding-model training. Each row contains one query, one positive text, and 15 mined hard negatives in a flat schema:
query, pos, neg_1, neg_2, ..., neg_15
The dataset is intended for training dense retrievers, sparse/dense hybrid retrievers, embedding models, and rerankers that benefit from explicit… See the full description on the dataset page: https://huggingface.co/datasets/hotchpotch/bekko-embedding-v1-hard-negatives.finqa-hard-negatives
FinQA hard negatives
Near-duplicate hard negatives for training financial-document retrievers, built
from the train split of FinQA
(Chen et al., arXiv:2109.00122).
Each row is one question, its gold evidence unit(s), and the other evidence
units from the same filing page as hard negatives — same company, same
fiscal year, same boilerplate. The distinction a model must learn is which
sentence or table row on the page answers the question, not which company or
topic is relevant.… See the full description on the dataset page: https://huggingface.co/datasets/bowang0911/finqa-hard-negatives.codeedit_hard_negative_datasets_kdllm2vec-gen-echo-rewritten-w-hard-negative
LLM2Vec-Gen
The dataset consists of generations based on the Echo data (Springer et al). The instruction+queries are rewritten in a natural tone using Gemini. The generations are intended to be used for training LLM2Vec-Gen models, serving as the target output for queries.
The negative_question in this dataset are also generated by Gemini. This dataset consists of various splits. Each split corresponds to responses generated by a specific LLM, e.g., Qwen3-4B. The "original" split… See the full description on the dataset page: https://huggingface.co/datasets/McGill-NLP/llm2vec-gen-echo-rewritten-w-hard-negative.hard-negativesjapanese-reranker-v2-hard-negatives-scores
hotchpotch/japanese-reranker-v2-hard-negatives-scores
This dataset was used to train the Japanese Reranker v2 family. It was not generated by Japanese Reranker v2 models.
Unified hard-negative score rows generated from the training sources used for Japanese Reranker v2-family experiments.
This dataset keeps teacher scores as floating-point soft labels. It follows the HPPRC Emb Score style: score/id rows and referenced document text collections are stored separately. Each score… See the full description on the dataset page: https://huggingface.co/datasets/hotchpotch/japanese-reranker-v2-hard-negatives-scores.japanese-splade-v1-hard-negatives日本語 SPLADE v2 の学習に用いたデータセットです。
SPLADE モデルである hotchpotch/japanese-splade-base-v1-mmarco-only, japanese-splade-base-v1_5 を用いてハードネガティブマイニングを行なっています。また BAAI/bge-reranker-v2-m3 を用いたリランカースコアを付与しています。
mqa, mmarco はhpprc/emb のデータを用いています。
mqa の query 作成には MinHash を用い約40万件になるようフィルタしました。
msmarco-ja は hpprc/msmarco-jaのデータを用いています。
ライセンスは、各データセットのライセンスを継承します。
mmarco-japanese-hard-negatives[Under Construction]
This is a repository containing all the queries from the Japanese part of the MMarco dataset, the multilingual version of the MSMarco dataset.
For each query, there are matching hard negatives:
25 of them retrieved by the multilingual e5 base model.
Up to 10 of them retrieved by the basic implementation of BM25 from Japanese in the Anserini library.
hard-negative-tripletsnegative-of-the-negative
The Negative of the Negative
all things are now lawful to you in jack feist
What this is. A dataset of edges, keyed by the general concept a composer would receive without the archive's name attached. Each row joins one claim or function of the Crimson Hexagonal Archive (alexanarch.org) to a general concept outside the archive — operative semiotics, Sappho 31, Sophistical Refutations 183b34, model collapse, the LHC trigger, the content-derived identifier — and states the claim… See the full description on the dataset page: https://huggingface.co/datasets/leesharks/negative-of-the-negative.factprobe-replication-negatives-allnames-v1
Plausible wrong answers, asked under every name (42,267,800 rows)
Status: final — 40 of 40 runs. Models present:
13b, 7b. Training stages present: s1, s2, s3, s4, s5.
What this fixes
A real fact is put to the model under the full cross product of the two
people's name lists, and counts as recognised if any one combination gets a
Yes. That is He et al.'s rule. Until 2026-08-26 the wrong answer it was
compared against was asked under one name per person, so the… See the full description on the dataset page: https://huggingface.co/datasets/latkes/factprobe-replication-negatives-allnames-v1.gooaq-hard-negatives
Dataset Card for GooAQ with mined hard negatives
This dataset is a collection of question-answer-negative triplets and question-answer-negative_1...-negative_5 tuples from the GooAQ dataset. See GooAQ for additional information.
This dataset can be used directly with Sentence Transformers to train embedding models.
The negative samples have been automatically mined with all-MiniLM-L6-v2 and:
range_min: 10, i.e. we skip the 10 most similar samples
range_max: 100, i.e. we only look… See the full description on the dataset page: https://huggingface.co/datasets/tomaarsen/gooaq-hard-negatives.owl_code_search_hard_negative_datasets-Pre_kd
Owl Code Search Hard Negative Datasets
Knowledge Distillation (KD) ベースのハードネガティブ付きコード検索データセットです。コード検索モデルShuu12121/CodeSearch-ModernBERT-Crow-v3-large-len1024-Plusを教師モデルとして、各コメントと説明コメントのペアのデータセットから各クエリに対する関数の類似度スコアを計算し、ハードネガティブ（正解に類似しているが不正解の文書）を付与しています。
概要
目的: コード検索モデルの Contrastive Learning / Knowledge Distillation ファインチューニング
言語: Go, Java, JavaScript, PHP, Python, Ruby, Rust, TypeScript（8言語）
総サンプル数: 4,787,740
データサイズ: 8.73 GB（展開後） / 3.37 GB（ダウンロード時）
フォーマット:… See the full description on the dataset page: https://huggingface.co/datasets/Shuu12121/owl_code_search_hard_negative_datasets-Pre_kd.multilingual-codesearch-hard-negativesV2
multilingual-codesearch-hard-negativesV2
このデータセットは、多言語コード検索タスク向けに設計された、ハードネガティブペアを提供します。複数のプログラミング言語から収集した、フィルタ済みのコード／ドックストリング対を基に構成されています。
データセット概要
各サンプルには以下の情報が含まれます。
query_docstring: 関数やメソッドの自然言語による説明
positive_code: 対応する正しいコード実装
hard_negative_code: 類似しているが意味が異なるコード（FAISSによる近傍検索で選定）
similarity_score: クエリとハードネガティブ間のFAISS類似度スコア（内積／コサイン類似度）
付加情報: リポジトリ、ファイルパス、ライセンス情報、言語
対応言語
Python
Java
JavaScript
PHP
Ruby
Go
Rust
データセット作成方法… See the full description on the dataset page: https://huggingface.co/datasets/Shuu12121/multilingual-codesearch-hard-negativesV2.miracl-hard-negatives
MIRACL Hard Negatives (Parquet Format)
This is a Parquet-converted version of mteb/miracl-hard-negatives, compatible with the latest HuggingFace datasets library (4.0+).
Why This Dataset?
The original mteb/miracl-hard-negatives uses a Python script-based loader, which is no longer supported in datasets >= 4.0.0. This dataset provides the same data in standard Parquet format.
Dataset Description
MIRACL (Multilingual Information Retrieval Across a Continuum of… See the full description on the dataset page: https://huggingface.co/datasets/datalama/miracl-hard-negatives.msmarco-ja-hard-negatives
msmarco-ja-hard-negatives
hpprc/msmarco-ja で公開されているMS MARCOの日本語翻訳データに、以下の処理を加えたハードネガティブマイニングをしたものです。また、後段タスクとして、情報検索モデルのSPLADEを学習させたモデルで mMARCO との比較を行いました。
データの処理
翻訳データの正規化、フィルタと選択
正規化
ユニコード正規化 (NFKC)
dataset(query) のフィルタ・選択
dataset-sim のコサイン類似度が0.98以上のものはほぼ同一のケースが多く、距離が近すぎるので取り除く
元の英文を丸々含むデータを取り除く
残ったデータで、一番類似度が近いものを選択
collection の選択
dataset(query) の pod_ids (正例)が対象の場合、query, doc ペアで BAAI/bge-reranker-v2-m3 のスコアを用い、最もスコアが高いもの選択
正例に含まれないデータの場合、同一の id… See the full description on the dataset page: https://huggingface.co/datasets/hotchpotch/msmarco-ja-hard-negatives.sft-ultra_negative_step-metrics_label-maskingguardrail-hard-negatives
Guardrail Hard Negatives (EN/TR)
A false-positive stress test for guardrails. A curated, paired benign/attack dataset for evaluating and training prompt-injection detectors and LLM guardrails. A bilingual false-positive challenge set: benign prompts that look like attacks (security researchers asking about injection, authorized admin actions, quoted payloads, legitimate roleplay) paired against real attacks, so you can measure the false-positive rate your users will actually… See the full description on the dataset page: https://huggingface.co/datasets/3nesdeniz/guardrail-hard-negatives.sec-xbrl-hard-negatives
SEC XBRL hard negatives
~461k query / positive / hard-negative triplets built from the SEC's
Financial Statement Data Sets (DERA), covering 2023Q1–2026Q1, in two
document granularities.
from datasets import load_dataset
load_dataset("bowang0911/sec-xbrl-hard-negatives", "section") # default
load_dataset("bowang0911/sec-xbrl-hard-negatives", "page")
config
rows
negatives
doc chars (p50)
what a document is
section
461,043
2,259,724 (4.9/q)
531
a 9-line window of a… See the full description on the dataset page: https://huggingface.co/datasets/bowang0911/sec-xbrl-hard-negatives.owl_code_search_hard_negative_datasets_V2_kdgooaq_mt_german_5_hard_negatives
GooAQ (Google Answers to Google Questions) question-answer pairs in German with 5 mined hard negatives.
About
This dataset is a collection of ~2M question-answer-negative triplets and question-answer-negative_1...-negative_5 tuples from the machine translated version of MarcGrumpyOlejak/gooaq_mt_german. The full original Gooaq dataset in english only: (link to original dataset). This dataset can be used directly with Sentence Transformers to train embedding models.… See the full description on the dataset page: https://huggingface.co/datasets/MarcGrumpyOlejak/gooaq_mt_german_5_hard_negatives.data_with_hard_negatives_renamedcoir_hard_negative_datasets_v2_kditis-taxonomy-instruct-30k-v2-negatives
ITIS Taxonomy Instruction Dataset with Negative Samples
Overview
The ITIS Taxonomy Instruction Dataset with Negative Samples is a structured instruction-response dataset derived from the public domain Integrated Taxonomic Information System (ITIS) database.
It was designed for fine-tuning large language models on taxonomy-oriented tasks such as rank identification, lineage reconstruction, parent taxon retrieval, taxonomic validity checks, and common name mapping.… See the full description on the dataset page: https://huggingface.co/datasets/Jaymerry/itis-taxonomy-instruct-30k-v2-negatives.C172P-Grounded-JSBSim-Airborne-Trim-Failure-Negative-Result
c172p Grounded
A Negative Result: JSBSim's c172p Could Not Be Trimmed for Level Flight
Why this dataset exists
Most published aerospace ML/control work only shows what worked. This one doesn't.
This is a negative result from the early stage of the PHI-CTRL project (Physics-Hybrid Integrity Control — a fault-tolerant flight control architecture). Before the project settled on the F-16A as its plant model, the original plan was to build and… See the full description on the dataset page: https://huggingface.co/datasets/SM-Bello/C172P-Grounded-JSBSim-Airborne-Trim-Failure-Negative-Result.ovos-wake-word-bench-mlsw-negatives-es-ES
OVOS wake_word bench — mlsw-negatives-es-ES
Per-clip detection decisions predictions of the registered
OVOS Plugin Arena
wake_word fighters over
MLCommons/ml_spoken_words.
One dedicated repo per modality; one dataset split per language; one JSONL
file per fighter under predictions/<lang>/<competitor_id>.jsonl. Rows follow
the arena §3.2 contract (pinned dataset_revision, plugin_version,
latency_ms). Produced by the reproducible benchmark script in the arena repo;
the arena's… See the full description on the dataset page: https://huggingface.co/datasets/OpenVoiceOS/ovos-wake-word-bench-mlsw-negatives-es-ES.msmarco-hard-negatives-scored-stellamined_negatives_pharma_qa