CoolFace
Modelpublic

Nixon/Phi-4-mini-instruct-WearableQA-GGUF

sourceHugging Facemitupdated 13d agoView on Hugging Face
0likes563downloads
Model Card

Phi-4-mini-instruct

This repository contains importance matrix (`imatrix`) quantized GGUF binaries of `microsoft/Phi-4-mini-instruct`.

To prevent loss of reasoning accuracy in domain-specific tasks, these models were quantized using an `imatrix` profile generated directly from domain tokens in the `facebook/WearableQA` dataset. This calibration optimizes weight retention across low-bit quants (IQ4_XS, IQ3_M, IQ2_XXS) specifically for health, physiological data, and wearable device reasoning.

📦 Model Files & Quantization

The following GGUF variants are available in this repository:

File NameQuant TypeSize (MB)BPWRecommended Use Case
phi4-mini-Q8_0.ggufQ8_0~3,895 MB8.00Precision baseline / server evaluation
phi4-mini-Q4_K_M.ggufQ4_K_M~2,378 MB4.50Standard desktop & local server deployment
phi4-mini-IQ4_XS.ggufIQ4_XS~2,121 MB4.25Recommended Edge Target (optimal perplexity/RAM ratio)
phi4-mini-IQ3_M.ggufIQ3_M~1,924 MB3.66Low-VRAM GPUs and mobile environments
phi4-mini-IQ2_XXS.ggufIQ2_XXS~1,244 MB2.06Ultra-constrained memory / wearable hardware
📁 Calibration File: domain.imatrix is included in this repository for reproducibility or generating additional quantization levels.

⚙️ Calibration & Build Details

  • —Base Model: microsoft/Phi-4-mini-instruct (3.8B parameters, 128k context window)
  • —Calibration Corpus: facebook/WearableQA (~60k domain-specific tokens)
  • —Prompt Formatting: Formatted via native tokenizer.apply_chat_template() matching Phi-4 Chat syntax.
  • —Compilation Tooling: Compiled natively via llama.cpp using -DGGML_CUDA=ON.
  • —Quantization Tool: llama-quantize with --imatrix domain.imatrix.

🚀 Quickstart & Usage

1. Run with llama.cpp CLI

bash
# Build or download llama.cpp binaries
./llama-cli \
  -m phi4-mini-IQ4_XS.gguf \
  -p "<|user|>\nPatient reports a sudden drop in HRV (22ms) and elevated skin temperature (+1.2°C). What metrics should be reviewed next?<|end|>\n<|assistant|>\n" \
  -n 256 \
  -c 2048 \
  -ngl 99 \
  --flash-attn on