datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
msmarco-qrels
Dataset Card for BEIR Benchmark
Dataset Summary
BEIR is a heterogeneous benchmark that has been built from 18 diverse datasets representing 9 information retrieval tasks:
Fact-checking: FEVER, Climate-FEVER, SciFact
Question-Answering: NQ, HotpotQA, FiQA-2018
Bio-Medical IR: TREC-COVID, BioASQ, NFCorpus
News Retrieval: TREC-NEWS, Robust04
Argument Retrieval: Touche-2020, ArguAna
Duplicate Question Retrieval: Quora, CqaDupstack
Citation-Prediction: SCIDOCS
Tweet… See the full description on the dataset page: https://huggingface.co/datasets/BeIR/msmarco-qrels.msmarco-yesnoms_marco_colbertv2
MS MARCO v1 Passage, ColBERTv2
Token-level (late-interaction) ColBERTv2 embeddings of the MS MARCO v1 passage collection and the dev/small queries.
Source
Collection: MS MARCO v1 passage (ir_datasets msmarco-passage), 8,841,823 passages
Queries: dev/small, 6,980 queries and 7,437 qrels (msmarco-passage/dev/small)
Document order: passage id order (row i is pid i)
Encoding
Model: ColBERTv2 (colbert-ir/colbertv2.0, BERT-base-uncased tokenizer)… See the full description on the dataset page: https://huggingface.co/datasets/tuskanny/ms_marco_colbertv2.tdist-msmarco-scores
MS MARCO Distillation Scores for Translate-Distill
This repository contains MS MARCO training
query-passage scores produced by MonoT5 reranker
unicamp-dl/mt5-13b-mmarco-100k and
castorini/monot5-3b-msmarco-10k.
Each training query is associated with the top-50 passages retrieved by the ColBERTv2 model.
Files are gzip compressed and with the naming scheme of {teacher}-monot5-{msmarco, mmarco}-{qlang}{plang}.jsonl.gz,
which indicates the teacher reranker that inferenced using… See the full description on the dataset page: https://huggingface.co/datasets/hltcoe/tdist-msmarco-scores.MS-ME-Detect-data
MS-ME-Detect paper-final reproduction bundle
This dataset repository contains the reproduction bundle for the MS-ME-Detect
paper-final model:
s_final = clip((1 - 0.108) * rank01(s_base) + 0.108 * rank01(s_qwen14_segment), 0, 1)
Final model: Qwen14 score-level late fusion between:
s_base: no-segment fused base score
s_qwen14_segment: embedding_segment_qwen14_probe__sgd_a1e4
External all_samples is included for final reporting only. It was not used
for training, candidate… See the full description on the dataset page: https://huggingface.co/datasets/Eromecc/MS-ME-Detect-data.ms_marco_cocondenserMS-MARCO-Passages
MS MARCO Passages — collection.tsv re-encoding
A verbatim mirror of Microsoft's MS MARCO Passage Ranking corpus, packaged as a single gzip-compressed tab-delimited file. The 8,841,823 rows, the integer passage_id column, and the passage text column match the upstream collection.tsv byte-for-byte — only the outer compression wrapper differs.
Re-hosted under Heliosoph for ingestion-pipeline stability — Microsoft's published archive (collection.tar.gz on… See the full description on the dataset page: https://huggingface.co/datasets/Heliosoph/MS-MARCO-Passages.msme-legal-dispute-classification-dataset
MSME Legal Dispute Classification Dataset
Overview
The MSME Legal Dispute Classification Dataset is a curated collection of legal dispute case documents categorized into six statutory dispute types under MSME-related contexts.
This dataset is designed for long-document multi-class legal text classification research and development.
It contains structured legal narratives including:
Statement of Claim
Buyer Response
Case Summary
Contractual and payment… See the full description on the dataset page: https://huggingface.co/datasets/maddyanand/msme-legal-dispute-classification-dataset.msmarco-pl-qrelsPart of BEIR-PL: Zero Shot Information Retrieval Benchmark for the Polish Language.
Link to arxiv: https://arxiv.org/pdf/2305.19840.pdf
Contact: konrad.wojtasik@pwr.edu.pl
India-Stock-Symbols-and-Metadata
India Stock Symbols & Company Metadata
This dataset contains stock symbols and basic company metadata for all listed companies in India.It is updated weekly if new changes are there.
📊 Dataset Contents
The dataset is provided as a CSV file with the following columns:
Column
Description
name
Full company name
ticker
Stock ticker symbol (e.g., AAPL, MSFT)
market
The exchange/market where the stock is listed
sector
The primary business sector of… See the full description on the dataset page: https://huggingface.co/datasets/msmisra1976/India-Stock-Symbols-and-Metadata.msme-legal-dispute-classification-dataset
MSME Legal Dispute Classification Dataset
Overview
The MSME Legal Dispute Classification Dataset is a curated collection of legal dispute case documents categorized into six statutory dispute types under MSME-related contexts.
This dataset is designed for long-document multi-class legal text classification research and development.
It contains structured legal narratives including:
Statement of Claim
Buyer Response
Case Summary
Contractual and payment details
The… See the full description on the dataset page: https://huggingface.co/datasets/abhinavdread/msme-legal-dispute-classification-dataset.msme-dispute-document-corpus
MSME Dispute Document Corpus (Synthetic OCR)
Dataset Description
This dataset contains 8,000+ synthetic document samples designed to train AI models for the Indian MSME (Micro, Small, and Medium Enterprises) dispute resolution sector.
It is specifically engineered to handle Real-World OCR Noise and Adversarial Edge Cases (e.g., distinguishing a "Proforma Invoice" from a valid "Tax Invoice"). The data mimics the messy, unstructured text often found in scanned PDFs, photos… See the full description on the dataset page: https://huggingface.co/datasets/abhinavdread/msme-dispute-document-corpus.ms_macro_bigMS-Marco-Prompt-generationmsmarco-sub
MS MARCO Subset
Purpose
This subset provides a standardized benchmark for evaluating sparse model performance on MS MARCO data, with negative examples pooled using BM25 retrieval.
Creation
This subset was created using the make_beir_subset.py script with the following command:
python ./make_beir_subset.py --dataset msmarco --es-host http://localhost:9200 --force-reindex --split dev
Parameters Used
Dataset: msmarco
Split: dev (development set)
ES… See the full description on the dataset page: https://huggingface.co/datasets/freethenation/msmarco-sub.MS_MARCO_Gold_Passage_QA
How to Make
Make "Question has only one answer".
If Question has multiple answer, that question is deleted.
Example Usage
from datasets import load_dataset
docs = load_dataset('jun000/MS_MARCO_Gold_Passage_QA')
ms_marco_inferencelessmsmarcoranking-querieskannolo-msmarco-splademsme-document-presence-dataset
MSME Document Presence Detection Dataset
Overview
This dataset is designed for training binary classification models to detect the presence of mandatory documents in MSME arbitration cases using OCR-extracted text.
The dataset supports automated document completeness validation systems.
Each sample represents a structured arbitration case with document-specific OCR text fields and binary presence labels.
Documents Covered
The dataset includes detection labels… See the full description on the dataset page: https://huggingface.co/datasets/abhinavdread/msme-document-presence-dataset.InRanker-msmarcomsmarco_passages_sampledmsmarco_train_hard_negativesThe dataset has 3 columns : query, positive_example, negative_example
First we filtered queries.train.tsv to only contain queries which were available in qrels.train.tsv. This was done so that we only take care of queries of which we had a positive example of. Then we found top 200 ranked passages in collection.tsv for each query in filtered queries and then randomly sampled a passage whose rank lied in the range of 50 to 200 to produce our final hard negatives for each query.
Use this as a… See the full description on the dataset page: https://huggingface.co/datasets/manupande21/msmarco_train_hard_negatives.msmarco_modifiedmsmarco_train_rankmsme-payment-dispute-dataset
MSME Payment Dispute Dataset
Description
This dataset contains structured case-level information for MSME payment disputes in India.
The dataset was constructed from structured extraction of:
MSME arbitration awards
Commercial court decisions
Public legal case repositories
All records are anonymized and structured for machine learning purposes.
Dataset Size
~4,600 structured cases
3 outcome classes:
win
settlement
escalation
Features… See the full description on the dataset page: https://huggingface.co/datasets/abhinavdread/msme-payment-dispute-dataset.kannolo-msmarco-cocondensermsmarco-shortCIIR_MSMARCOmulti-msmarco