Modelpublic

permutans/Tarka-Embedding-150M-V1-ONNX

sourceHugging Facegemmaupdated 9mo agoView on Hugging Face

1likes8downloads

Model Card

Tarka-Embedding-150M-V1 (ONNX)

ONNX version of Tarka-AIR/Tarka-Embedding-150M-V1.

—Embedding dimension: 768
—Context length: 2048
—Model size: ~600MB

Usage

With ONNX Runtime (Python)

python

import onnxruntime as ort
from transformers import AutoTokenizer

session = ort.InferenceSession("tarka-150m-v1-onnx/model.onnx")
tokenizer = AutoTokenizer.from_pretrained("permutans/Tarka-Embedding-150M-V1-ONNX")

texts = [
    "The weather is lovely today.",
    "It's so sunny outside!",
    "He drove to the stadium.",
]

embeddings = []
for text in texts:
    inputs = tokenizer(text, return_tensors="np")
    _, sentence_embedding = session.run(
        None,
        {"input_ids": inputs["input_ids"], "attention_mask": inputs["attention_mask"]},
    )
    embeddings.append(sentence_embedding[0])

import numpy as np

embeddings = np.array(embeddings)
print(embeddings.shape)  # (3, 768)

# Compute cosine similarities
from sklearn.metrics.pairwise import cosine_similarity

similarities = cosine_similarity(embeddings)
print(similarities)

With FastEmbed (Rust)

Compatible with fastembed-rs for high-performance embedding generation.

Model Outputs

—token_embeddings: Token-level embeddings (batchsize, sequencelength, 768)
—sentence_embedding: Pooled sentence embeddings (batch_size, 768) - use this for most tasks

Performance

This ONNX export works with both CPU and CUDA execution providers for flexible deployment.