Datasetpublic

hotchpotch/NanoBEIR-es-with-bm25

🚧 This dataset is currently under construction. Specifications may change. NanoBEIR-es (with bm25 subset) A BM25-first retrieval subset derived from lightonai/NanoBEIR-es for reproducible NanoBEIR evaluation. What this dataset is Language: es BM25 top-k: 100 Number of evaluation splits: 13 This folder contains generated BM25 retrieval results and reproducibility metadata. Data structure corpus: _id, text queries: _id, text qrels: query-id… See the full description on the dataset page: https://huggingface.co/datasets/hotchpotch/NanoBEIR-es-with-bm25.

sourceHugging Faceupdated 7mo agoView on Hugging Face

0likes241downloads

Dataset Card

🚧 This dataset is currently under construction. Specifications may change.

NanoBEIR-es (with bm25 subset)

A BM25-first retrieval subset derived from lightonai/NanoBEIR-es for reproducible NanoBEIR evaluation.

What this dataset is

—Language: es
—BM25 top-k: 100
—Number of evaluation splits: 13
—This folder contains generated BM25 retrieval results and reproducibility metadata.

Data structure

—corpus: _id, text
—queries: _id, text
—qrels: query-id, corpus-id, score
—bm25: query-id, corpus-ids (A bm25 subset is included for first-stage retrieval and reranking experiments.)

BM25 baseline scores

Scope	nDCG@10	nDCG@100
13-split average	0.5074	0.5518

Selected BM25 algorithm

Item	Value
Mode	`nltk_stem`
Details	NLTK snowball stemming

Subset names

—Split names: NanoArguAna, NanoClimateFEVER, NanoDBPedia, NanoFEVER, NanoFiQA2018, NanoHotpotQA, NanoMSMARCO, NanoNFCorpus, NanoNQ, NanoQuoraRetrieval, NanoSCIDOCS, NanoSciFact, NanoTouche2020
—Config names in this output folder: bm25

BM25 split-level baseline (`nDCG`)

Split	nDCG@10	nDCG@100
`NanoArguAna`	0.3930	0.4468
`NanoClimateFEVER`	0.2847	0.3686
`NanoDBPedia`	0.6207	0.6457
`NanoFEVER`	0.7878	0.8023
`NanoFiQA2018`	0.3269	0.3942
`NanoHotpotQA`	0.7525	0.7743
`NanoMSMARCO`	0.4114	0.4733
`NanoNFCorpus`	0.3006	0.2252
`NanoNQ`	0.3280	0.4086
`NanoQuoraRetrieval`	0.7933	0.8122
`NanoSCIDOCS`	0.2975	0.4019
`NanoSciFact`	0.7173	0.7341
`NanoTouche2020`	0.5830	0.6865

Upstream source

—Original dataset: lightonai/NanoBEIR-es

License

Same as the upstream dataset. Follow upstream licensing and attribution requirements.