Modelpublic

oli-mebberson/quark-1.5b-GGUF

sourceHugging Facemitupdated 5d agoView on Hugging Face

0likes166downloads

Model Card

Lattice Quark 1.5B GGUF

GGUF builds of Lattice Quark 1.5B, a decoder-only language model trained from scratch by Oli Mebberson. It is not a fine-tune of another base model.

Quark was pretrained on about 2B tokens, then instruction-tuned for chat. These files package the released SFT checkpoint for llama.cpp and local apps.

Quants

File	Size	Tested identity answer	Use case
`quark-1.5b-sft-Q4_K_M.gguf`	892,415,072 bytes (0.85 GiB)	Generic assistant answer	Smallest practical build
`quark-1.5b-sft-Q5_K_M.gguf`	1,053,338,720 bytes (0.98 GiB)	Generic assistant answer	More weight fidelity than Q4KM
`quark-1.5b-sft-Q6_K.gguf`	1,224,320,096 bytes (1.14 GiB)	Generic assistant answer	Higher-fidelity quant
`quark-1.5b-sft-Q8_0.gguf`	1,585,439,840 bytes (1.48 GiB)	Preserves the trained Lattice Quark answer	Recommended when model identity matters
`quark-1.5b-sft-f16.gguf`	2,983,322,688 bytes (2.78 GiB)	Preserves the trained Lattice Quark answer	Reference build

The trained answer to "Who are you?" is unusually sensitive to quantization. In greedy tests, only Q80 and f16 answered as Lattice Quark. Q6K, Q5KM and Q4KM stayed coherent but switched to a generic assistant identity. Gravity explanations and Fibonacci code stayed coherent in tested Q4KM, Q5KM and Q8_0 builds.

The simple arithmetic weakness is still present. The model can ramble or answer incorrectly, even at Q8_0. Treat that as a model limitation rather than a reason to assume a smaller quant is broken.

Important: patched llama.cpp required

Quark uses a custom nanochat architecture. Stock llama.cpp does not currently load these files.

This repo includes:

—quark-llama-cpp.bundle, a cloneable git bundle containing the tested llama.cpp base and nanochat support commit
—quark-nanochat-llamacpp.patch, the standalone architecture patch
—convert_to_gguf.py, the converter used for this release
—NOTES.md, build, integration and verification notes

To restore the tested llama.cpp tree:

bash

git clone quark-llama-cpp.bundle llama.cpp
cd llama.cpp
cmake -B build -DGGML_METAL=ON -DLLAMA_CURL=OFF
cmake --build build --config Release -j

Then run a GGUF with the chat template embedded in the file:

bash

./build/bin/llama-cli \
  -m quark-1.5b-sft-Q8_0.gguf \
  -cnv \
  -c 2048

Model details


Parameters	1.5B
Architecture	GPT-style nanochat, 26 layers, 1536 embedding width, 12 heads
Context	2048 tokens
Vocabulary	32,768, tiktoken-style BPE
Pretraining	About 2B tokens of the SmolLM corpus, 1,500 iterations
Instruction tune	465 iterations using `<	user_start	> ... <	assistant_start	>`
Training	From random initialization, then SFT

The raw checkpoints and original model card are in the source model repo. The training code is at olii-dev/nano-gpt.

Verification

The conversion was checked at several levels:

—tokenizer output matched the source tokenizer byte-for-byte on eight test strings
—a small end-to-end reference test matched all 24 of 24 greedy token IDs and produced mean NLL 6.2510 in llama.cpp versus 6.2498 in the reference
—every published quant loaded and generated text in the patched llama.cpp build
—the f16 and Q8_0 builds reproduced the trained Lattice Quark identity answer
—SHA-256 checksums were verified after the artifacts were copied to storage

Limits

Quark is a small model trained in the open. It can hold a conversation, follow instructions and write simple code, but it makes arithmetic errors, can wander, and may confidently state false information. Do not use it for decisions that need reliable factual or numerical answers.

More Lattice models and demos are at lattice.mebbo.cloud.