Seneelya/harrier-oss-v1-0.6b-GGUF
Harrier OSS v1 600M — GGUF
Tested and converted using llama.cpp b10054+
GGUF conversion of microsoft/harrier-oss-v1-0.6b for use with llama.cpp.
Microsoft Harrier OSS v1 600M is a compact multilingual text embedding model built on the Gemma3 architecture, producing 1024-dimensional embeddings using last-token pooling and L2 normalization.
Files
Usage
llama-server --embeddings -m harrier-oss-v1-0.6b-q8_0.gguf --port 8080Then query the /embedding endpoint as with any llama.cpp embedding model.
⚠️ Found conversion issues and my fixes
As of conversion time (July 2026), the stock convert_hf_to_gguf.py from llama.cpp could not convert this model out of the box. If you hit the same errors converting similar Gemma3-based embedding models, here's how they were resolved.
1. NotImplementedError: BPE pre-tokenizer was not recognized
chkhsh: fc65e8f033752f900de442edbaa0e385712ff5f6f1d2fd8d958fa6ec59daa92fThis tokenizer's hash wasn't yet in the list of known pre-tokenizers hardcoded in get_vocab_base_pre(). Fix: add an entry for it (in convert_hf_to_gguf.py / conversion/base.py, depending on your llama.cpp version):
if chkhsh == "fc65e8f033752f900de442edbaa0e385712ff5f6f1d2fd8d958fa6ec59daa92f":
# ref: microsoft/harrier-oss-v1 (gemma3-based BPE tokenizer)
res = "llama-bpe"2. AssertionError on assert max(tokenizer.vocab.values()) < vocab_size
The tokenizer declares an id exactly 1 higher than the number of rows actually present in token_embd.weight. Do not raise `vocab_size` in config.json to work around this — doing so desyncs the GGUF metadata (n_vocab) from the actual embedding matrix row count, and will fail to load with:
check_tensor_dims: tensor 'token_embd.weight' has wrong shape; expected 640, 262145, got 640, 262144, 1, 1Instead, drop the extra id and keep the original vocab_size. Patch for get_vocab_base():
def get_vocab_base(self) -> tuple[list[str], list[int], str]:
tokens: list[str] = []
toktypes: list[int] = []
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(self.dir_model)
vocab_size = self.hparams.get("vocab_size", len(tokenizer.vocab))
extra_ids = {tid: tok for tok, tid in tokenizer.vocab.items() if tid >= vocab_size}
if extra_ids:
logger.warning(
f"tokenizer has {len(extra_ids)} id(s) >= vocab_size ({vocab_size}) with no embedding row, "
f"dropping them: {extra_ids}"
)
tokpre = self.get_vocab_base_pre(tokenizer)
reverse_vocab = {id_: encoded_tok for encoded_tok, id_ in tokenizer.vocab.items() if id_ < vocab_size}
added_vocab = tokenizer.get_added_vocab()
added_tokens_decoder = tokenizer.added_tokens_decoder
for i in range(vocab_size):
if i not in reverse_vocab:
tokens.append(f"[PAD{i}]")
toktypes.append(gguf.TokenType.UNUSED)
else:
token: str = reverse_vocab[i]
if token in added_vocab:
if not added_tokens_decoder[i].normalized:
previous_token = token
token = tokenizer.decode(tokenizer.encode(token, add_special_tokens=False))
if previous_token != token:
logger.info(f"{repr(previous_token)} is encoded and decoded back to {repr(token)} using AutoTokenizer")
if added_tokens_decoder[i].special or self.does_token_look_special(token):
toktypes.append(gguf.TokenType.CONTROL)
else:
token = token.replace(b"\xe2\x96\x81".decode("utf-8"), " ")
toktypes.append(gguf.TokenType.USER_DEFINED)
else:
toktypes.append(gguf.TokenType.NORMAL)
tokens.append(token)
return tokens, toktypes, tokpreQuality check
Not formally benchmarked against the original PyTorch model — feel free to open a discussion if you run comparisons.
License
MIT, inherited from microsoft/harrier-oss-v1-0.6b.