puttatidam/microbiology1k-mya-qaretrieval

microbiology1k-mya-qaretrieval

Deduplicated copy of `kornwtp/microbiology1k-mya-qaretrieval`, part of the SEA-BED data-quality work.

• —Source dataset: kornwtp/microbiology1k-mya-qaretrieval
• —Deduplicated on: 2026-09-17
• —Task type: qa_retrieval
• —Splits: train

What changed

Kept in this dataset's ORIGINAL schema -- same columns, same nesting, same extra fields (ids, titles, answers) -- so it is a drop-in replacement for the source repo. Documents differing only in case/whitespace were folded onto one canonical surface; repeated queries were deduplicated; documents listed as both positive and negative were resolved in favour of positive; queries left with no positive document were dropped. Families whose positives are a list column (miracl, mldr, wangchanx-legalrag) merge a repeated query's gold sets; the one-context-per-row families keep the first occurrence instead, because their schema cannot record more than one gold document per query. The query_policy column in the summary CSV states which was used.

Row counts and the exact policy used are recorded in qa_retrieval_dedup_summary.csv in the analysis repo. This dataset is not identical to the published SEA-BED data -- scores computed on it are not directly comparable to published SEA-BED numbers.