CoolFace
Modelpublic

liodon-ai/LocateAnything-3B-imatrix-GGUF

sourceHugging Faceotherupdated 3mo agoView on Hugging Face
1likes424downloads
Model Card

LocateAnything-3B — iMatrix GGUF

iMatrix GGUF quantizations of [nvidia/LocateAnything-3B](https://huggingface.co/nvidia/LocateAnything-3B) — the first GGUF available for this model.

LocateAnything-3B is NVIDIA's 3B visual grounding model — it locates and identifies objects in images given natural language descriptions. Designed for on-device deployment and robotics applications.

These GGUFs use importance matrix (iMatrix) calibration on 2M tokens of wikitext-103: iMatrix runs calibration text through the model, measures which weights activate most, and protects them during quantization. Result: noticeably better coherence at Q2/Q3/Q4 — same file size, better output.

Quick Start

llama.cpp

bash
llama-cli -hf liodon-ai/LocateAnything-3B-imatrix-GGUF:Q4_K_M

LM Studio / Jan

Search liodon-ai/LocateAnything-3B-imatrix-GGUF and pick your quant.

Available Quants

QuantSizeVRAMNotes
IQ2_M1.28 GB2 GBultra-tiny + iMatrix — better than standard Q2
IQ3_M1.62 GB2.5 GBtiny + iMatrix — sharper than standard Q3
IQ4_XS1.91 GB3 GBsmall + iMatrix — rivals Q5 at Q4 size
Q2_K1.38 GB2 GBtiniest standard — runs almost anywhere, iMatrix-improved
Q3_K_M1.73 GB2.5 GBgreat for 4GB VRAM, iMatrix-improved
Q4_K_M2.11 GB3 GBsweet spot (recommended), iMatrix-improved
Q5_K_M2.44 GB4 GBhigh quality, iMatrix-improved
Q6_K2.80 GB4 GBnear-lossless, iMatrix-improved
Q8_03.62 GB5 GBbasically full quality

What is iMatrix?

Standard quantization rounds all weights equally. iMatrix:

  1. 1.Runs calibration text through the full-precision model
  2. 2.Measures which weights activate most (the "importance matrix")
  3. 3.Allocates more precision to important weights, less to unimportant ones

Same file size. Better output. Most noticeable at Q2/Q3/Q4.

Calibration

Importance matrix computed from 2M tokens of wikitext-103 — 128 calibration chunks.

Source Model

  • —Original: nvidia/LocateAnything-3B
  • —Architecture: 3B visual grounding model
  • —Strengths: Object localization, visual grounding, on-device robotics
  • —License: NVIDIA Open Model License