Modelpublic

Hcompany/NeoMME-260M-Retriever-ST-dense

sourceHugging Faceapache-2.0updated 15d agoView on Hugging Face

4likes579downloads

Model Card

NeoMME-Retriever (260M): Single-Tower Multimodal-Native Multilingual Foundation Encoder 🔎

[!IMPORTANT] NeoMME-Retriever (260M) variants: - Default (`transformers`): Returns dense and multi-vector embeddings together with a single forward pass. Recommended for most use cases and inference. - ST dense [current]: Supports independent dense fine-tuning with Sentence Transformers. - ST late-interaction: Supports independent multi-vector fine-tuning with Sentence Transformers.

![Hugging Face](https://huggingface.co/docs/transformers/en/model_doc/neomme) ![Hugging Face](https://hf.co/collections/Hcompany/neomme) ![arXiv](https://arxiv.org/abs/2609.01657)

NeoMME-260M-Retriever-ST-dense is a model for multimodal document retrieval. Fine-tuned from NeoMME-260M, it encodes text queries and documents (text or page screenshots) using one shared bidirectional Transformer encoder.

This model can be used with Sentence Transformers, but can only generate dense embeddings.

<table> <thead> <tr style="background-color: rgba(146, 81, 247, 0.20);"><th>Specification</th><th>Value</th></tr> </thead> <tbody> <tr><td>Parameters</td><td>263M</td></tr> <tr><td>Vocabulary</td><td>131,072 tokens</td></tr> <tr><td>Context length</td><td>16,384 tokens</td></tr> <tr><td>Hidden size</td><td>1,024</td></tr> <tr><td>Image patches</td><td>32 × 32 pixels, up to 2,048 pixels on the longest side (default)</td></tr> <tr><td>Dense embeddings</td><td>1,024 dimensions (Matryoshka: [128, 256, 512, 1,024])</td></tr> <tr><td>Dense pooling strategy</td><td>Mean</td></tr> </tbody> </table>

Dense embeddings are L2-normalized and use cosine similarity. They match NeoMMEForRetrieval.dense_embeddings.

Performance

All scores use the metric shown at the full trained dimensions. Higher is better. ViDoRe v3, v2, and v1 measure visual document retrieval, while BEIR-15 measures text retrieval.

<table> <thead> <tr><th rowspan="2">Benchmark</th><th rowspan="2">Metric</th><th colspan="2" style="background-color: rgba(146, 81, 247, 0.20);">NeoMME-260M</th><th colspan="2">NeoMME-800M</th></tr> <tr><th>Late interaction</th><th style="background-color: rgba(146, 81, 247, 0.20);">Dense [current]</th><th>Late interaction</th><th>Dense</th></tr> </thead> <tbody> <tr><td>ViDoRe v3</td><td>nDCG@10</td><td>0.5226</td><td style="background-color: rgba(37, 99, 235, 0.20);">0.3907</td><td>0.5560</td><td>0.4391</td></tr> <tr><td>ViDoRe v2</td><td>nDCG@5</td><td>0.5218</td><td style="background-color: rgba(37, 99, 235, 0.20);">0.4075</td><td>0.5591</td><td>0.4475</td></tr> <tr><td>ViDoRe v1</td><td>nDCG@5</td><td>0.8598</td><td style="background-color: rgba(37, 99, 235, 0.20);">0.7552</td><td>0.8744</td><td>0.7993</td></tr> <tr><td>BEIR-15</td><td>nDCG@10</td><td>0.4881</td><td style="background-color: rgba(37, 99, 235, 0.20);">0.3055</td><td>0.5126</td><td>0.3686</td></tr> </tbody> </table>

Usage

bash

pip install -U "sentence-transformers[image]"

python

from sentence_transformers import SentenceTransformer


model = SentenceTransformer("Hcompany/NeoMME-260M-Retriever-ST-dense")

queries = [
    "Quelle partie de la production pétrolière du Kazakhstan provient de champs en mer ?",
    "Which hour of the day had the highest overall electricity generation in 2019?",
]
documents = [
    "https://github.com/tonywu71/colpali-cookbooks/blob/main/examples/data/shift_kazakhstan.jpg?raw=true",
    "https://github.com/tonywu71/colpali-cookbooks/blob/main/examples/data/energy_electricity_generation.jpg?raw=true",
]

query_embeddings = model.encode_query(queries, convert_to_tensor=True)
document_embeddings = model.encode_document(documents, convert_to_tensor=True)
scores = model.similarity(query_embeddings, document_embeddings)

# Expected: scores[0, 0] > scores[0, 1] and scores[1, 1] > scores[1, 0].
print(scores)

The score tensor has shape (num_queries, num_documents) and scores[i, j] is the score between query i and document j. A larger value indicates a closer match.

Training

NeoMME-260M-Retriever was fine-tuned from NeoMME-260M on text retrieval and document-page images. Training uses a joint late-interaction and Matryoshka dense contrastive objective.

The NeoMME technical report describes the full fine-tuning recipe.

Limitations

With Sentence Transformers, only one of the two retrieval heads can be used at a time.

License

Model weights are released under the Apache 2.0 license.

Citation

bibtex

@misc{lac2026neommesingletowermultimodalnativemultilingual,
      title={NeoMME: A Single-Tower Multimodal-Native Multilingual Foundation Encoder for Efficient Fine-Tuning and Inference},
      author={Aurélien Lac and Tony Wu},
      year={2026},
      eprint={2609.01657},
      archivePrefix={arXiv},
      primaryClass={cs.IR},
      url={https://arxiv.org/abs/2609.01657},
}