Hanno-Labs/dinghy-law-8b-v1-gguf

dinghy-law-8b-v1 GGUF

GGUF / llama.cpp build of `Hanno-Labs/dinghy-law-8b-v1`, a multilingual legal text-embedding model with 8B parameters, derived from `nvidia/Nemotron-3-Embed-8B-BF16`. Source revision `8a2d25a2c845f0e3001e3198c71b2a0db4eee27e`.

Files

Q8_0 is the recommended default when memory permits. Q6_K trades a smaller footprint for modest additional quantization error. Q4_K_M is the compact option; use the measured fidelity below when choosing.

Runtime geometry

This is a non-causal Ministral3 bidirectional encoder embedding model with mask-aware mean pooling, 4096-dimensional L2-normalized embeddings, and an 8192-token published context cap. The conversion preserves the source model's is_causal: false setting in GGUF metadata.

The GGUF tokenizer metadata sets automatic BOS insertion to false and automatic EOS insertion to false. These values match the source tokenizer; do not add special tokens manually.

The source Sentence Transformers package ends with a 4096-dimensional Dense projection. llama.cpp does not carry that post-pooling projection, so this release uses the backbone geometry: Transformer -> mean pool -> L2 normalize. On the frozen multilingual legal probes, cosine between the full HF package and the backbone-only representation is 0.99638 mean / 0.99251 minimum. The table below separately reports each quant against both the backbone and the full public model.

Queries must begin with query: and documents with passage: . These are semantic roles used during training and evaluation, not optional display labels.

Fidelity on frozen legal probes

The probe set includes query and document roles in English, German, and Chinese plus an 8192-token legal-document probe. The release job must pass its declared per-quant thresholds before any repository is made public.

Usage with llama.cpp

Query embedding:

llama-embedding -m dinghy-law-8b-v1-Q8_0.gguf \
  -p "query: What is the punishment for criminal breach of trust by a public servant?" \
  --pooling mean --embd-normalize 2 -c 8192

Document embedding:

llama-embedding -m dinghy-law-8b-v1-Q8_0.gguf \
  -p "passage: Whoever commits criminal breach of trust may be punished under the applicable statute." \
  --pooling mean --embd-normalize 2 -c 8192

Evaluation reference

The full Sentence Transformers model scores 72.58 Mean(Task) nDCG@10 on MTEB(Law, v1). That benchmark number describes the full HF package; the GGUF variants are documented by direct embedding-fidelity measurements above.

Provenance

Converted from the immutable source revision with the official llama.cpp converter and quantizer. Converter source commit: 5f55650a7. Container: ghcr.io/ggml-org/llama.cpp@sha256:c44fb21e7342c21d22f74516aafb47294bf5b5886a929ccc67f9325c67b39b9d.

The legal training data and adaptation process are public in the source model repository. License: openmdw-1.1. The complete license text and origin notices are included in LICENSE and NOTICE.