Modelpublic

nabin2004/nebium-small-gguf

sourceHugging Facemitupdated 5d agoView on Hugging Face

0likes87downloads

Model Card

Nebium-Small-GGUF

Quantized and FP16 GGUF format binaries for Nebium-Small (117M parameters).

Designed for low-latency CPU and GPU execution with llama.cpp and Ollama.

—PyTorch Base Model: nabin2004/nebium-small
—GGUF Repository: nabin2004/nebium-small-gguf
—Source Repository: github.com/nabin2004/nebium

Artifacts

Filename	Precision	Description
`nebium-small.gguf`	FP16	Full-precision baseline export
`tokenizer.json`	Tokenizer	BPE vocabulary and merge definitions

Inference with llama.cpp

bash

# Clone and compile llama.cpp
git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && make

# Download GGUF binary
huggingface-cli download nabin2004/nebium-small-gguf nebium-small.gguf --local-dir .

# Run prompt continuation
./llama-cli -m nebium-small.gguf -p "e2e4 e7e5 g1f3" -n 25 --temp 0.7

Inference with Ollama

dockerfile

# Modelfile
FROM ./nebium-small.gguf
PARAMETER temperature 0.7
PARAMETER stop "<|eos|>"
SYSTEM You are an autoregressive chess next-move prediction model using UCI move notation.

bash

ollama create nebium-small -f Modelfile
ollama run nebium-small "e2e4 e7e5"

License

MIT License.