SuperPauly/harrier-oss-v1-0.6b-gguf

harrier-oss-v1-0.6B GGUF

This repository contains GGUF exports and quantized variants of `microsoft/harrier-oss-v1-0.6b`, a multilingual text embedding model from Microsoft.

These files are intended for llama.cpp-compatible runtimes that support embedding models in GGUF format. The original upstream repository remains the source of truth for the native Transformers and Sentence Transformers checkpoints, training details, and canonical examples.

Model Summary

• —Base model: microsoft/harrier-oss-v1-0.6b
• —Model type: multilingual text embedding model
• —Parameters: 0.6B
• —Embedding dimension: 1024
• —Max context length: 32768 tokens
• —Pooling: last-token pooling
• —Normalization: L2 normalization
• —Languages: 94 languages
• —License: MIT

Harrier OSS v1 models are designed for retrieval, semantic similarity, clustering, classification, bitext mining, and reranking workloads. This 0.6B variant is the mid-sized model in the Harrier OSS v1 family.

Available Files

Benchmark Results

The table below reformats the local benchmark results that were already present in this repository. The values are preserved exactly and grouped by quantization for easier comparison on Hugging Face.

Benchmark source: llama-bench build 6307ec07d (8604).

These are local throughput measurements, not embedding-quality scores. Expect different throughput on different CPUs, GPUs, thread counts, and llama.cpp revisions.

Usage Notes

This repository packages an embedding model, not a chat or text-generation model.

• —Use a llama.cpp-compatible runtime with embedding support.
• —Queries should include a short task instruction, following the upstream training format.
• —Documents and passages should usually be encoded without an instruction prefix.
• —Match the source model behavior by using last-token pooling and normalized embeddings if your runtime exposes those controls.

Minimal llama.cpp server example:

llama-server -m harrier-oss-v1-0.6B-Q4_K_M.gguf --embedding

Example query text format:

Instruct: Given a web search query, retrieve relevant passages that answer the query
Query: summit define

For the original Transformers and Sentence Transformers examples, refer to the upstream model card:

• —https://huggingface.co/microsoft/harrier-oss-v1-0.6b

Source Model Details

According to the upstream Microsoft model card, Harrier OSS v1 uses a decoder-only architecture with last-token pooling and L2 normalization to produce dense text embeddings. The model family is designed for multilingual retrieval and related embedding tasks, and the 0.6B release reports an MTEB v2 score of 69.0.

Notes

• —This repo only contains GGUF artifacts derived from the upstream model.
• —No benchmark rows were added for BF16 because there were no BF16 benchmark results in the original local README.md.
• —If you need the original checkpoint files, prompts, or training background, use the upstream Microsoft repository.

Acknowledgements

• —Original model: Microsoft Harrier OSS v1
• —GGUF ecosystem and runtime support: llama.cpp