NoemaAI-labs/Noema-2B-GGUF

Noema-2B — GGUF

GGUF builds of **NoemaAI-labs/Noema-2B**, a ~2B reasoning model post-trained from Qwen/Qwen3.5-2B (hybrid Gated-DeltaNet architecture), for use with llama.cpp and compatible runtimes.

For the full model description, benchmarks, training recipe, and limitations, see the **base model card**.

⚠️ Requires Qwen3.5 (qwen35) architecture support in llama.cpp

This model uses the Qwen3.5 hybrid Gated-DeltaNet architecture. It will only load in a llama.cpp build that includes the qwen35 architecture (linear + full attention layers). On an older build you will get an "unknown architecture" / unsupported-arch error. Use a recent llama.cpp with Qwen3.5 support and build from source if your package manager's version is too old.

Files

All quants were produced from the same F16 conversion of the release weights and smoke-tested (load + generation) on each file.

Usage

Noema is tuned and evaluated in non-thinking mode, which is the default the embedded chat template applies. Pass --jinja so that template is used, and the model runs in its trained (non-thinking) regime automatically — no extra flags or prompt tags are needed.

Recommended sampling (the config Noema was evaluated under): --temp 0.7 --top-p 0.8 --top-k 20 --presence-penalty 1.5. Avoid `--temp 0` / greedy decoding — this model family is documented to enter repetition loops under greedy sampling.

# One-shot, single-turn (non-thinking):
llama-cli -m Noema-2B-Q4_K_M.gguf --jinja -cnv -st \
  -p "A farmer has 17 red apples and 25 green apples. How many in total?" \
  -n 256 --temp 0.7 --top-p 0.8 --top-k 20 --presence-penalty 1.5

# Or run an OpenAI-compatible server (set the sampling params client-side,
# or pass the same flags here as server defaults):
llama-server -m Noema-2B-Q4_K_M.gguf --jinja -c 4096 \
  --temp 0.7 --top-p 0.8 --top-k 20 --presence-penalty 1.5

License

Apache-2.0, inherited from the Qwen/Qwen3.5-2B base model.