sentence-corpus
msmarco-corpus
MS MARCO Corpus
This dataset allows for a convenient mapping from MS MARCO query/passage ID to the query/passage text. This passage corpus was downloaded from https://msmarco.z22.web.core.windows.net/msmarcoranking/collection.tar.gz, and the queries from https://msmarco.blob.core.windows.net/msmarcoranking/queries.tar.gz (via Wayback Machine).
Usage
This dataset was designed to allow you to perform the following:
from datasets import load_dataset
query_dataset =… See the full description on the dataset page: https://huggingface.co/datasets/sentence-transformers/msmarco-corpus.smclm-10M-sentence-corpus@ARTICLE{11068992,
author={Perełkiewicz, Michał and Dadas, Sławomir and Poświata, Rafał},
journal={IEEE Access},
title={SMCLM: Semantically Meaningful Causal Language Modeling for Autoregressive Paraphrase Generation},
year={2025},
volume={},
number={},
pages={1-1},
keywords={Semantics;Transformers;Training;Encoding;Decoding;Syntactics;Measurement;Computational modeling;Autoencoders;Adaptation models;Autoregressive models;paraphrase generation;text embeddings;semantically… See the full description on the dataset page: https://huggingface.co/datasets/mmichall/smclm-10M-sentence-corpus.english-news-sentence-corpus