PollardWeights/FrogMini-14B-Pollard

FrogMini-14B — Pollard

### Pollard shrank this model: 29.54 GB (f16) → 4.56 GB — 85% smaller, 6.5× down. The smallest rung here; larger, higher-fidelity rungs are listed below. | format | this model's size | |---|---:| | f16 | 29.54 GB | | Q80 | ~15.66 GB | | Q6K | ~12.11 GB | | Q4KM | ~8.57 GB | | PollardMix (this repo's IQ2_XXS) | 4.56 GB |

Pollard builds of microsoft/FrogMini-14B-2510 made with Pollard Weights — a ladder of measured-allocation quants (bits placed by per-layer sensitivity, not a uniform crush).

Standard GGUF — runs in stock llama.cpp / ik_llama.cpp, Ollama, LM Studio. Trellis (IQ*_KT) files need ik_llama.cpp; the K-quants run anywhere.

Model details

Which file should I choose?

Every rung is the same weights, sized to a different RAM budget by the measured allocation. Pick the largest one that fits your machine with room for context:

• —~14 GB RAM / VRAM → `Q6_K` (12.12 GB). near-lossless
• —~10 GB RAM / VRAM → `IQ4_XS` (8.43 GB). recommended default
• —~9 GB RAM / VRAM → `IQ3_S` (6.79 GB). best size/quality trade
• —~8 GB RAM / VRAM → `IQ2_S` (5.94 GB). small
• —~7 GB RAM / VRAM → `IQ2_XXS` (4.56 GB). smallest - 6.5x down from f16

Available files (wikitext-2 test, ctx 512)

f16 reference PPL 9.3589.

tok/s measured on an RTX 5070 Ti (16 GB), full GPU offload.

Sampling

Every rung cleared the coherence gate on the first sampling config -- three prompts including code, no loops, down to and including IQ2_XXS. Ship these defaults:

--temp 0.7 --repeat-penalty 1.15 --repeat-last-n 256 --top-k 40 --top-p 0.9

About the bottom rung

IQ2_XXS is coherent, and it is also a real step down: +3.91 PPL against f16, where every rung above it costs under a point. It exists so a 14B fits in 4.56 GB. If you have the room, IQ3_S is 2 GB larger and gives most of the quality back.

Prompt format

ChatML

Download a specific file

pip install -U "huggingface_hub[cli]"
hf download PollardWeights/FrogMini-14B-Pollard \
  --include "FrogMini-14B-Pollard-IQ4_XS.gguf" --local-dir ./

How to run

These are standard GGUF and run with llama.cpp:

llama-server -hf PollardWeights/FrogMini-14B-Pollard:IQ4_XS

or from a local file:

llama-cli    -m FrogMini-14B-Pollard-IQ4_XS.gguf -ngl 99 -p "Explain why the sky is blue."
llama-server -m FrogMini-14B-Pollard-IQ4_XS.gguf -ngl 99      # OpenAI-compatible API + web UI at :8080

They also work in anything built on llama.cpp — LM Studio, koboldcpp, Jan, ramalama, Ollama (ollama run hf.co/PollardWeights/FrogMini-14B-Pollard).

imatrix (calibration)

The importance matrix (FrogMini-14B-Pollard.imatrix, included) was computed on a Calib 3.0 multi-domain corpus (prose, code, math, multilingual), 40 chunks.

ARM / AVX

llama.cpp repacks weights into an interleaved layout at load time for faster inference on ARM and AVX machines — no special file needed, online repacking covers these quants. The old Q4_0_4_4/4_8/8_8 variants are not required.

Errata

• —Trellis (IQ*_KT) quants need ik_llama.cpp to build/run; K-quants run in any recent llama.cpp.
• —Measured allocation places bits by per-layer sensitivity under a size budget.
• —Single machine; replication invited.

Credits & license

• —Base model: `microsoft/FrogMini-14B-2510` (Microsoft)
• —Quantization tooling: llama.cpp (ggml-org)
• —Method + tooling: Pollard Weights — measure first, no claim before a number.
• —License: mit, inherited from the base model.

Built with [Pollard Weights](https://github.com/WestWaters/pollard-weights) — frontier models, small hardware, no compromise.