Modelpublic

ibm-granite/granite-8b-code-instruct-4k-GGUF

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face

13likes3kdownloads

Model Card

⚠️ DEPRECATION WARNING ⚠️

⚠️ NOT RECOMMENDED FOR USE IN NEW PROJECTS ⚠️

New applications/projects should use the latest mainline Granite language model family, whose code capabilities supercede this model. This model is being made available strictly for historical/scientific purposes. Please see our [Granite Collections](https://huggingface.co/ibm-granite/collections) for the latest Granite releases.

image/png

ibm-granite/granite-8b-code-instruct-4k-GGUF

This is the Q4KM converted version of the original `ibm-granite/granite-8b-code-instruct-4k`. Refer to the original model card for more details.

Use with llama.cpp

shell

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp

# install
make

# run generation
./main -m granite-8b-code-instruct-4k-GGUF/granite-8b-code-instruct.Q4_K_M.gguf -n 128 -p "def generate_random(x: int):" --color