CoolFace
Datasetpublic

tuskanny/nfcorpus_lateon

NFCorpus, LateOn Token-level (late-interaction) embeddings of the BEIR NFCorpus corpus and queries, encoded with LateOn, in the TACHIOM multivector format. Source BEIR NFCorpus, test split. Corpus, queries and qrels were read from the official BEIR files via ir_datasets (beir/nfcorpus/test); PyLate only did the encoding 3,633 documents, 323 queries, 12,334 qrels Text given to the encoder for each document: title + " " + text (BEIR title and body joined by a… See the full description on the dataset page: https://huggingface.co/datasets/tuskanny/nfcorpus_lateon.

sourceHugging Faceupdated 8h agoView on Hugging Face
0likes21downloads
Dataset Card

NFCorpus, LateOn

Token-level (late-interaction) embeddings of the BEIR NFCorpus corpus and queries, encoded with LateOn, in the TACHIOM multivector format.

Source

  • —BEIR NFCorpus, test split. Corpus, queries and qrels were read from the official BEIR files via ir_datasets (beir/nfcorpus/test); PyLate only did the encoding
  • —3,633 documents, 323 queries, 12,334 qrels
  • —Text given to the encoder for each document: title + " " + text (BEIR title and body joined by a space). The text itself is not included, only its vectors
  • —Row order follows the BEIR corpus and query files; row i of doc_ids.npy / queries_ids.npy identifies row i of doclens.npy / queries.npy

Encoding

  • —Model: `lightonai/LateOn` @ 62911e105059585d244384c7d17826e35f669c17
  • —Library: PyLate 1.6.0, CPU
  • —Document length cap: 300 tokens (model default)
  • —Query length: 32 tokens (model default)
  • —Query expansion: no (model default)
  • —The model defaults come from config_sentence_transformers.json in the model repository. We did not override any of them
  • —Documents: punctuation tokens are dropped (PyLate skiplist), as are padding tokens
  • —Vectors: 128-d, L2-normalized

Statistics

Token vectors (N)862,599
Avg vectors per document237.4 (max 286)
Vectors per queryvariable, 4 to 25 (no query expansion), zero-padded to 25
Avg vectors per query8.6

Files

FiledtypeShapeContent
documents.npyfloat16 (<f2)[862599, 128]All document vectors, concatenated document by document
doclens.npyint32[3633]Vectors per document; sum == N
token_ids.npyuint32[862599]Input token id of each row of documents.npy
doc_ids.npystring[3633]BEIR doc id of each document
queries.npyfloat32[323, 25, 128]Query vectors, zero-padded at the end
query_lens.npyint32[323]True number of vectors per query
queries_ids.npystring[323]BEIR query id of each query
qrels.test.tsvTREC12334 linesqid \t 0 \t docid \t relevance
groundtruth/gt_top100.tsvTSV32300 linesExhaustive top-100: query_idx \t doc_idx \t rank \t score (0-based positions)
groundtruth/gt_ids.npyint32[323, 100]Same, as doc positions
groundtruth/gt_scores.npyfloat32[323, 100]Same, MaxSim scores

Zero padding does not change any score: a zero query vector adds 0 to the MaxSim of every document.

Exhaustive-search effectiveness

Exact MaxSim over the full collection (vectorium compute_groundtruth_multivec). These are the reference numbers for approximate search on this data.

nDCG@10R@100
0.38090.3366