Modelpublic

end000/Spark-X2.5-4B-Q8_0-GGUF

sourceHugging Faceapache-2.0updated 24d agoView on Hugging Face

4likes2.8kdownloads

Model Card

Spark-X2.5-4B Q8_0 GGUF

Q8_0 quantization of XHToken/Spark-X2.5-4B, converted from the official BF16 GGUF with the compatible XHToken/llama.cpp implementation.

File

—Spark-X2.5-4B-Q8_0.gguf — Q8_0, approximately 4.38 GB.

llama.cpp server example

bash

llama-server -m ./Spark-X2.5-4B-Q8_0.gguf --alias spark-x2.5-4b -c 65536 -ngl all -fit off -fa on -ctk q4_0 -ctv q4_0 --jinja --temp 1.0 --top-p 0.95 --top-k 0 -np 1 --host 127.0.0.1 --port 8081

Spark-X2.5 support requires the compatible XHToken llama.cpp implementation linked above.

License

Apache License 2.0, following the base model.