Modelpublic

danbev/hailo-llama-3.2-GGUF

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face

2likes27downloads

Model Card

hailo-llama-3.2 Model Card

Description

This is an experimental GGUF model for Hailo hailo-llama-3.2 in llama.cpp.

Usage

You can use this model with the following branch: hailo.

Clone and build:

console

$ git clone -b hailo https://github.com/danbev/llama.cpp.git && cd llama.cpp
$ cmake -B build -S . -DGGML_HAILO=ON -DGGML_NATIVE=ON
$ cmake --build build --target llama-cli -j 12

Run:

console

$ ./build/bin/llama-cli -hf danbev/hailo-llama-3.2-GGUF