cstr/most-embed-de-GGUF

most-embed-de GGUF

GGUF format of malteos/most-embed-de for use with CrispEmbed.

MOST Embed DE — German customer-support retrieval model, with query: / passage: prompts and normalized 2048-d embeddings.

Files

Parity vs HuggingFace reference

Cosine similarity vs the upstream sentence-transformers reference on a fixed test set (text):

Note: below the 0.99 retrieval-quality bar — text: q4_k (0.987). Embeddings are still functionally usable (>0.9 = directionally correct for similarity ranking) but expect small differences in nearest-neighbor results vs the upstream f32 reference.

License and provenance

The fine-tune is distributed under CC-BY-NC-4.0; commercial use is not permitted without separate authorization from the fine-tune author. It is derived from NVIDIA's Nemotron-3-Embed-1B-BF16, whose Model Materials are distributed under OpenMDW-1.1. Redistribution must retain the OpenMDW agreement plus all applicable copyright and origin notices. Both sets of terms and the upstream model cards must be reviewed and preserved.

Quick Start

# Download
huggingface-cli download cstr/most-embed-de-GGUF most-embed-de-q4_k-attn-q8.gguf --local-dir .

# Run with CrispEmbed
./crispembed -m most-embed-de-q4_k-attn-q8.gguf "Hello world"

# Or with auto-download
./crispembed -m most-embed-de "Hello world"

Model Details

Verification

Compared with the original Transformers implementation. F16 reaches cosine 1.000000 at every dumped transformer boundary and on the final embedding. Q80 reaches final cosine 0.999818. The compact Q4K artifact keeps token embeddings and attention at Q8_0; over eight German query/document texts it has minimum cosine 0.987191, preserves every top-1 retrieval result, and reduces maximum similarity-score error to 0.02595.

Usage with CrispEmbed

CrispEmbed is a lightweight C/C++ text embedding inference engine using ggml. No Python runtime, no ONNX. Supports BERT, XLM-R, Qwen3, and Gemma3 architectures.

# Build CrispEmbed
git clone https://github.com/CrispStrobe/CrispEmbed
cd CrispEmbed
cmake -S . -B build && cmake --build build -j

# Encode
./build/crispembed -m most-embed-de-q4_k-attn-q8.gguf "query text"

# Server mode
./build/crispembed-server -m most-embed-de-q4_k-attn-q8.gguf --port 8080
curl -X POST http://localhost:8080/v1/embeddings \
    -d '{"input": ["Hello world"], "model": "most-embed-de"}'

Credits

• —Original model: malteos/most-embed-de
• —Inference engine: CrispEmbed (ggml-based)
• —Conversion: convert-decoder-embed-to-gguf.py