hotchpotch/mmarco-hard-negatives-reranker-filtered

mMARCO Reranker-Filtered Hard Negatives (Multilingual)

Overview

This dataset is built from mMARCO (multilingual MS MARCO) triplets for each language subset. For each (query, positive), hard negatives are bundled and then filtered using cross-encoder re-scoring. The goal is to remove negatives that are too strong or incorrect for training. The same procedure is applied to all language subsets.

The dataset is published as mmarco-hard-negatives-reranker-filtered with config names {lang}-{variant}. {lang} is the language subset name (e.g., japanese), and {variant} is one of the following. The pair format is not included in the public release.

1) {lang}-hard-negatives

The filtered hard negatives as-is. Columns: query: str, pos_text: str, negs_text: list[str], negs_count: int, pos_score: float, negs_score: list[float]

2) {lang}-triplet

For each (query, pos_text), one negative is randomly selected and converted into a (query, positive, negative) triplet. Columns: query: str, positive: str, negative: str

3) {lang}-triplet-10

For each (query, pos_text), up to 10 negatives are randomly sampled, and each is expanded into a (query, positive, negative) triplet. Columns: query: str, positive: str, negative: str

4) {lang}-triplet-all

All negatives in negs_text are expanded into (query, positive, negative) triplets. Columns: query: str, positive: str, negative: str

5) {lang}-hard-negatives-7

Only records with at least 7 negatives are kept. Then 7 negatives are randomly selected and stored as negative_1..negative_7. Columns: query: str, positive: str, negative_1: str, negative_2: str, negative_3: str, negative_4: str, negative_5: str, negative_6: str, negative_7: str

Columns: query: str, positive: str, negative_1: str, negative_2: str, negative_3: str, negative_4: str, negative_5: str, negative_6: str, negative_7: str

Source data

• —Dataset: unicamp-dl/mmarco, a multilingual version of MS MARCO
• —Revision: refs/convert/parquet (parquet-converted version)
• —Target subsets: all language subsets available under refs/convert/parquet
• —Split: partial train Parquet for each language ({lang}/partial/train/*.parquet or {lang}/partial-train/*.parquet)
• —Main columns in source: query, positive, negative

Construction procedure (reproducible processing)

The following steps reproduce the dataset. We describe the processing itself rather than local scripts or environments.

1. Aggregate triplets into hard-negative bundles

1. 1.Load all partial train Parquet files for each language subset.
2. 2.Keep only rows where query, positive, and negative are all present.
3. 3.Group by (query, positive) and deduplicate negatives with a set.
4. 4.For each (query, positive), create a record:
5. 5.query: string
6. 6.pos_text: positive
7. 7.negs_text: unique list of negatives for that (query, positive) (sorted for determinism)

2. Cross-encoder re-scoring

Score (query, text) pairs using:

• —Model: BAAI/bge-reranker-v2-m3 (Cross-Encoder)
• —Max length: 512 tokens
• —No quantization or distillation; standard inference in bf16

For each record:

1. 1.Score (query, pos_text) → pos_score
2. 2.Score (query, neg) for each negs_text → negs_score (same order as negs_text)

3. Filtering conditions

The reranker-score filtering here is implemented with reference to the approach in ruri-v3-dataset-reranker. Keep a record only if all conditions hold:

• —pos_score > 0.3
• —keep only negatives with neg_score < 0.7
• —at least 1 negative remains after filtering

Save the remaining negative count as negs_count.

Output columns

• —query (string)
• —pos_text (string)
• —negs_text (list[string])
• —negs_count (int)
• —pos_score (float)
• —negs_score (list[float])

negs_score follows the same order as negs_text.

License

This dataset follows the licenses and usage terms of the original sources. The immediate upstream dataset is unicamp-dl/mmarco, which is derived from MS MARCO. Downstream users should review the mMARCO and MS MARCO license/terms before use. The reranker-filtered hard-negative release is not relicensed separately from the upstream data.