nanoandrew4/Nemotron-3-Embed-1B-GGUF

Nemotron-3-Embed-1B GGUF

GGUF conversion of nvidia/Nemotron-3-Embed-1B-BF16 for use with llama.cpp.

Files

Usage

Serve with llama-server for embeddings (mean pooling, L2 normalized output):

llama-server -m model-BF16.gguf \
  --pooling mean \
  --embed-normalize 2 \
  --host 0.0.0.0 --port 8080

Then query:

curl http://localhost:8080/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{"input": "hello world"}'

Notes

• —Architecture: mistral3 (Ministral3, 16 layers, hidden 2048, 24 Q / 8 KV heads, 262k context with YaRN).
• —The model was trained and served as a sentence-transformers model, which does not prepend a BOS token. llama.cpp's tokenizer for this model does add one by default. If you need embeddings that exactly match the Hugging Face reference, run the server with: --override-kv tokenizer.ggml.add_bos_token=bool:false Retrieval quality is unaffected either way (the BOS token applies a nearly constant offset to all embeddings), but the raw vectors will differ.
• —Converted with convert_hf_to_gguf.py from the BF16 safetensors.