Nick-Maximillien/nexus-forensic-medgemma-4b.Q8_0.gguf

🛡️ Nexus-Forensic-MedGemma-4B (GGUF)

[image]

The Edge-Optimized Structural Compiler for Computable Medical Law

The System-Level Structural Compiler for Knowledge Base Construction

This repository contains the quantized GGUF weights for Nexus-Forensic-MedGemma-4B.

It is a fine-tuned LoRA adapter merged into MedGemma-1.5-4b-it, optimized to function as a system-callable tool for offline protocol compilation and Knowledge Graph generation.

Model Summary

• —Architecture: Gemma-3 (4B Parameters)
• —Quantization: Q8_0 (8-bit Integer Quantization)
• —Primary Task: Neurosymbolic Program Synthesis (Clinical Prose → Executable JSON)
• —Deployment Target: Regional healthcare facilities with limited or no cloud connectivity (Edge Deployment)

Quantization Fidelity & Latency Profile

Transitioning from 16-bit BrainFloat (FP16) to 8-bit Integer (Q8_0) quantization was a strategic decision to enable Edge Sovereignty without compromising forensic accuracy.

Performance Comparison

Technical Note The negligible decrease in schema validation confirms that the LoRA-stabilized attention-to-syntax ratio is preserved during 8-bit linear quantization.

Quantization Context (The Build)

This artifact was generated using the MedGemma-Edge-Converter-V2 pipeline.

The process involved:

1. 1.Physical fusion of LoRA weights into the 16-bit base model
2. 2.8-bit quantization using the llama.cpp engine

Engine Parameters

• —Merge Logic: Physical fusion of adapter weights into google/medgemma-1.5-4b-it
• —Quantization Backend: llama.cpp (v3+)
• —Target Precision: Q8_0 (Optimal balance of speed and forensic precision)

## Edge-Specific Optimization: KV-Cache Stabilization

To mitigate structural drift common in smaller, quantized models running on limited RAM, we implemented KV-Cache Stabilization during the physical fusion process.

The Issue

Standard quantization often causes “token stuttering” in JSON keys when running on low-thread CPUs.

The Fix

The MedGemma-Edge-Converter-V2 uses a custom calibration set of MoH clinical guidelines during the llama.cpp quantization pass to ensure that 8-bit weights prioritize high-probability structural tokens (e.g., {, ", :).

Production Hardware & Latency Tiers

Evaluations were conducted using llama-cpp-python with a 2048 token context window.

Installation & Setup

Option 1: Standalone CLI Run (llama.cpp)

For developers running the model directly on a local machine.

Install llama.cpp

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make

Download the GGUF

Place medgate_brain_4b_Q8.gguf in the models/ directory.

Run Inference (Standalone)

./main -m models/medgate_brain_4b_Q8.gguf \
  -p "### Instruction:\nYou are a Forensic Logic Parser...\n\n### Input:\nFollowing an acute anterior MI..." \
  -n 512

Option 2: Integrated System Run (Nexus Forensic)

Run the model inside the full Nexus Forensic Django/Docker ecosystem.

Clone the System

git clone https://github.com/Nick-Maximillien/nexus-forensic.git
cd nexus-forensic

Place the GGUF

Move medgate_brain_4b_Q8.gguf to the project root (same directory as manage.py).

Configure Environment

Set the following environment variable:

OFFLINE_EDGE=True

Run the System

docker-compose up --build

Developer Implementation (Python)

For custom integrations, use llama-cpp-python. The model follows the Alpaca Instruction Format for deterministic structural compilation.

from llama_cpp import Llama

# Initialize the Edge Engine
llm = Llama(
    model_path="medgate_brain_4b_Q8.gguf",
    n_ctx=2048,
    n_threads=4,  # Optimized for Docker / WSL2
    verbose=False
)

# Instruction Prompt
prompt = """### Instruction:
You are a Forensic Logic Parser. Convert the following clinical guideline text into an executable JSON schema.

### Input:
All pregnant women in areas of moderate to high malaria transmission receive intermittent preventive treatment (IPTp).

### Response:
"""

output = llm(
    prompt,
    max_tokens=1024,
    temperature=0.0,
    stop=["<|endoftext|>", "###"]
)

print(output["choices"][0]["text"])

Jurisdiction & Safety Note

• —Enforces Ministry of Health (MoH) Kenya, NASCOP, and KQMH clinical standards
• —Intended for post-hoc forensic auditing and protocol compilation
• —Not designed for real-time clinical diagnosis
• —Must be used as a deterministic logic gate within a regulated system

Project Resources

• —Adapter Repository: Adapter
• —Kaggle Notebook: MedGemma Structural Compiler Architecture
• —System Repository: Github system repo
• —Live Dashboard: nexus-forensic.vercel.app