datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
msmarco-qrels
Dataset Card for BEIR Benchmark
Dataset Summary
BEIR is a heterogeneous benchmark that has been built from 18 diverse datasets representing 9 information retrieval tasks:
Fact-checking: FEVER, Climate-FEVER, SciFact
Question-Answering: NQ, HotpotQA, FiQA-2018
Bio-Medical IR: TREC-COVID, BioASQ, NFCorpus
News Retrieval: TREC-NEWS, Robust04
Argument Retrieval: Touche-2020, ArguAna
Duplicate Question Retrieval: Quora, CqaDupstack
Citation-Prediction: SCIDOCS
Tweet… See the full description on the dataset page: https://huggingface.co/datasets/BeIR/msmarco-qrels.msmarco-yesnoms_marco_colbertv2
MS MARCO v1 Passage, ColBERTv2
Token-level (late-interaction) ColBERTv2 embeddings of the MS MARCO v1 passage collection and the dev/small queries.
Source
Collection: MS MARCO v1 passage (ir_datasets msmarco-passage), 8,841,823 passages
Queries: dev/small, 6,980 queries and 7,437 qrels (msmarco-passage/dev/small)
Document order: passage id order (row i is pid i)
Encoding
Model: ColBERTv2 (colbert-ir/colbertv2.0, BERT-base-uncased tokenizer)… See the full description on the dataset page: https://huggingface.co/datasets/tuskanny/ms_marco_colbertv2.MS-ME-Detect-data
MS-ME-Detect paper-final reproduction bundle
This dataset repository contains the reproduction bundle for the MS-ME-Detect
paper-final model:
s_final = clip((1 - 0.108) * rank01(s_base) + 0.108 * rank01(s_qwen14_segment), 0, 1)
Final model: Qwen14 score-level late fusion between:
s_base: no-segment fused base score
s_qwen14_segment: embedding_segment_qwen14_probe__sgd_a1e4
External all_samples is included for final reporting only. It was not used
for training, candidate… See the full description on the dataset page: https://huggingface.co/datasets/Eromecc/MS-ME-Detect-data.ms_marco_cocondenserkannolo-msmarco-splademsmarco-pl-qrelsPart of BEIR-PL: Zero Shot Information Retrieval Benchmark for the Polish Language.
Link to arxiv: https://arxiv.org/pdf/2305.19840.pdf
Contact: konrad.wojtasik@pwr.edu.pl
msme-dispute-document-corpus
MSME Dispute Document Corpus (Synthetic OCR)
Dataset Description
This dataset contains 8,000+ synthetic document samples designed to train AI models for the Indian MSME (Micro, Small, and Medium Enterprises) dispute resolution sector.
It is specifically engineered to handle Real-World OCR Noise and Adversarial Edge Cases (e.g., distinguishing a "Proforma Invoice" from a valid "Tax Invoice"). The data mimics the messy, unstructured text often found in scanned PDFs, photos… See the full description on the dataset page: https://huggingface.co/datasets/abhinavdread/msme-dispute-document-corpus.msmarco-sub
MS MARCO Subset
Purpose
This subset provides a standardized benchmark for evaluating sparse model performance on MS MARCO data, with negative examples pooled using BM25 retrieval.
Creation
This subset was created using the make_beir_subset.py script with the following command:
python ./make_beir_subset.py --dataset msmarco --es-host http://localhost:9200 --force-reindex --split dev
Parameters Used
Dataset: msmarco
Split: dev (development set)
ES… See the full description on the dataset page: https://huggingface.co/datasets/freethenation/msmarco-sub.ms_marco_inferencelessmsme-document-presence-dataset
MSME Document Presence Detection Dataset
Overview
This dataset is designed for training binary classification models to detect the presence of mandatory documents in MSME arbitration cases using OCR-extracted text.
The dataset supports automated document completeness validation systems.
Each sample represents a structured arbitration case with document-specific OCR text fields and binary presence labels.
Documents Covered
The dataset includes detection labels… See the full description on the dataset page: https://huggingface.co/datasets/abhinavdread/msme-document-presence-dataset.InRanker-msmarcomsmarco_modifiedmsme-payment-dispute-dataset
MSME Payment Dispute Dataset
Description
This dataset contains structured case-level information for MSME payment disputes in India.
The dataset was constructed from structured extraction of:
MSME arbitration awards
Commercial court decisions
Public legal case repositories
All records are anonymized and structured for machine learning purposes.
Dataset Size
~4,600 structured cases
3 outcome classes:
win
settlement
escalation
Features… See the full description on the dataset page: https://huggingface.co/datasets/abhinavdread/msme-payment-dispute-dataset.kannolo-msmarco-cocondenserCIIR_MSMARCOmsmacro