Lorelum/granite-embedding-97m-multilingual-r2-GGUF

Granite Embedding 97M Multilingual R2 — Q4_0 GGUF

Q4_0 quantization of IBM Granite Embedding 97M Multilingual R2, prepared for Lorelum's local CPU embedding service. This is a derivative artifact maintained by Lorelum, not an official IBM release.

File

• —Filename: granite-q4_0.gguf
• —Size: 66,345,216 bytes
• —SHA-256: 18e8ce8ce834790618e90d26bed465cca87362076f3042eb0d8eee0732596f59
• —Embedding dimensions: 384
• —Lorelum uses CLS pooling and L2 normalization, with no query or document prefix.

Provenance and quantization

Original model: ibm-granite/granite-embedding-97m-multilingual-r2.

F16 GGUF conversion by ATF:

• —Source file: granite-embedding-97m-multilingual-r2-f16.gguf
• —Source size: 206,403,072 bytes
• —Source SHA-256: 74075aeea7bd9ac4e5d74755e216fa487f5b6cce9ac6c5a363936370585b1a38

Quantized from that F16 file, without requantization, using llama.cpp b10901, commit 28ff0958291ce3465fabd7bd679d4b0edd742bd9:

llama-quantize --pure --token-embedding-type q4_0 \
  granite-embedding-97m-multilingual-r2-f16.gguf \
  granite-q4_0.gguf Q4_0 4

Matrix weights and the token embedding table use Q40; normalization vectors remain F32. This is the pure Q40 artifact, not Q4KM.

Usage

Download with the Hugging Face CLI:

hf download Lorelum/granite-embedding-97m-multilingual-r2-GGUF \
  granite-q4_0.gguf --local-dir ./models

Load it with a compatible llama.cpp embedding runtime. Lorelum validates the artifact size and SHA-256 before loading it. This artifact was tested with CPU inference on macOS arm64; publication alone does not establish compatibility or performance on other hardware.

License and attribution

The original IBM model and ATF conversion declare Apache-2.0. This repository includes the Apache-2.0 LICENSE, the upstream model card, and the source GGUF model card to retain source documentation and attribution. The modification made here is Q4_0 quantization; no additional training was performed.