COE-CBT-AI/Divyansh_XRPCS-Q4_K_M-GGUF

dtripathi18/Gemma3XRPCS-Q4K_M-GGUF

This is a quantized GGUF version of Gemma3_XRPCS, a vision-language model fine-tuned to generate layman-language summaries of chest X-ray (CXR) findings for patient-facing use.

Model Details

• —Base model: Gemma3-4B Vision (3.88B parameters)
• —Fine-tuning method: QLoRA (parameter-efficient fine-tuning, 4-bit NF4)
• —Training data: PadChest-derived professional-to-layman report pairs, generated via the BioLaySumm 2025 shared task pipeline
• —Quantization: Q4KM (GGUF)
• —Context length: 131,072 tokens

Intended Use

This model generates patient-accessible, plain-language summaries of chest X-ray findings from a paired radiograph and/or clinical report. It is intended as a research prototype demonstrating the feasibility of on-device, compact vision-language models for this task.

Use with llama.cpp

Install llama.cpp through brew (works on Mac and Linux):

brew install llama.cpp

CLI

llama-cli --hf-repo dtripathi18/Gemma3_XRPCS-Q4_K_M-GGUF --hf-file gemma3_xrpcs-q4_k_m.gguf -p "Summarize the key findings of this chest X-ray in plain language."

Server

llama-server --hf-repo dtripathi18/Gemma3_XRPCS-Q4_K_M-GGUF --hf-file gemma3_xrpcs-q4_k_m.gguf -c 2048

You can also use this checkpoint directly via the usage steps in the llama.cpp repo:

Step 1: Clone llama.cpp from GitHub.

git clone https://github.com/ggerganov/llama.cpp

Step 2: Move into the llama.cpp folder and build it with the LLAMA_CURL=1 flag along with hardware-specific flags (e.g., LLAMA_CUDA=1 for Nvidia GPUs on Linux).

cd llama.cpp && LLAMA_CURL=1 make

Step 3: Run inference through the main binary.

./llama-cli --hf-repo dtripathi18/Gemma3_XRPCS-Q4_K_M-GGUF --hf-file gemma3_xrpcs-q4_k_m.gguf -p "Summarize the key findings of this chest X-ray in plain language."

Citation

If you use this model, please cite: