Modelpublic

AaryanK/GLM-4.6V-Flash-GGUF

sourceHugging Facemitupdated 5mo agoView on Hugging Face

1likes591downloads

Model Card

GLM-4.6V-Flash-GGUF

This repository contains GGUF format quantizations of [zai-org/GLM-4.6V-Flash](https://huggingface.co/zai-org/GLM-4.6V-Flash).

Model Introduction

GLM-4.6V-Flash is a lightweight multimodal model (9B parameters) optimized for local deployment and low-latency applications, part of the GLM-V family. It features a 128k context window and achieves state-of-the-art performance in visual understanding among models of similar scale.

Key features include:

—9B Parameters (MoE): Optimized for efficiency.
—Native Multimodal Function Calling: Can process images/screenshots directly as tool inputs.
—Interleaved Image-Text Generation: Supports complex multimodal contexts.
—Document Understanding: Processes up to 128K tokens of multi-document inputs.

Usage

Note: This model uses the Glm4vMoe architecture. Please ensure you are using the latest version of llama.cpp to ensure compatibility.

Example with llama.cpp

bash

./llama-cli -m GLM-4.6V-Flash-Q4_K_M.gguf --mmproj GLM-4.6V-Flash-mmproj-model-f16.gguf -p "Describe this image" --image your_image.jpg