Hanno-Labs/dinghy-law-0.6b-v1-gguf

dinghy-law-0.6b-v1 — GGUF

GGUF / llama.cpp build of `Hanno-Labs/dinghy-law-0.6b-v1`, a compact (0.6B) legal text-embedding model (contrastive fine-tune of Qwen/Qwen3-Embedding-0.6B). Source revision 8f63ca78b621eb9c242429ea2472389d23f8902f.

What this is (backbone-only)

The sentence-transformers pipeline is Transformer -> last-token Pooling -> Dense -> L2 Normalize. llama.cpp carries the backbone: Transformer -> last-token pool -> L2 norm. The 2_Dense head is dropped — it is a near-orthogonal rotation (normalized ortho-defect 0.00103, unit row-norms, ~zero bias), and cosine similarity is invariant under rotation, so dropping it does not change retrieval ranking. The f16 GGUF reproduces the HF backbone embedding at cosine 1.00000 on legal probes.

Files

• —dinghy-law-0.6b-v1-f16.gguf — 1.20 GB
• —dinghy-law-0.6b-v1-Q8_0.gguf — 0.64 GB
• —dinghy-law-0.6b-v1-Q6_K.gguf — 0.49 GB

Recommended: `Q8_0` (near-lossless, best default). Q6_K for a smaller footprint with negligible quality loss. f16 as the full-precision reference.

Fidelity — cosine(HF backbone, GGUF) on legal probes

Worst min-cos across quants: 0.99009 (PASS).

Usage (llama.cpp)

Embeddings use last-token pooling and L2 normalization:

llama-embedding -m dinghy-law-0.6b-v1-Q8_0.gguf \
  -p "Retrieve statutes governing landlord obligations for habitability." \
  --pooling last --embd-normalize 2

For retrieval, format the query with the Qwen3-Embedding instruction prefix (documents are embedded raw):

Instruct: {task_instruction}\nQuery: {query}

e.g. task instruction Identify the most relevant statutes for the given situation. for statute retrieval.

Provenance

Quantized from the source safetensors with llama.cpp/convert_hf_to_gguf.py (f16) + llama-quantize (Q80, Q6K). MTEB(Law, v1) Mean(Task) nDCG@10 of the full model = 65.83. License: apache-2.0.