Modelpublic

Novadata-Technologies/Reason-ModernColBERT-ONNX-8bit

sourceHugging Facemitupdated 6mo agoView on Hugging Face

0likes

Model Card

Reason-ModernColBERT-ONNX-8bit

ONNX export of lightonai/Reason-ModernColBERT for fast CPU inference.

Model Details

—Source Model: lightonai/Reason-ModernColBERT
—Embedding Dimension: 128
—Format: ONNX (FP32 + INT8)

Files

File	Description
`model.onnx`	FP32 ONNX model
`model_int8.onnx`	INT8 quantized model (faster)
`tokenizer.json`	Tokenizer configuration
`config_sentence_transformers.json`	Model configuration

Usage with colbert-onnx (Rust)

rust

use colbert_onnx::Colbert;

let mut model = Colbert::from_pretrained("path/to/model")?;
let embeddings = model.encode_documents(&["Hello world"])?;

Export Tool

This model was exported using pylate-onnx-export:

bash

pip install "pylate-onnx-export @ git+https://github.com/lightonai/next-plaid.git#subdirectory=onnx/python"
pylate-onnx-export lightonai/Reason-ModernColBERT --push-to-hub Novadata-Technologies/Reason-ModernColBERT-ONNX-8bit