BrandeisPatrick/Ouro-1.4B-GGUF

Ouro-1.4B — GGUF

GGUF conversions of ByteDance/Ouro-1.4B, a looped language model: the entire 24-layer decoder stack is applied 4 times per token with shared weights, so a 1.4B-parameter model computes at an effective depth of 96 layers.

What this repository is, and is not

These are the first GGUFs of this architecture. llama.cpp had no ouro architecture, so no GGUF runtime could load Ouro at all — the request on the Ollama tracker (#14252) has been open since February 2026. The architecture was written for this release; the patch and the evaluation harness are at BrandeisPatrick/loop-transformer.

These files need a patched llama.cpp today. The ouro architecture is not yet in upstream llama.cpp, so stock llama.cpp, Ollama and LM Studio cannot load them yet. Build with the patch: git clone https://github.com/BrandeisPatrick/loop-transformer && loop-transformer/llamacpp/build.sh. Once the upstream PR merges and Ollama bumps its llama.cpp pin, these files will run unmodified.

Files

GSM8K is a fixed 200-problem subset (100 for the quants), 3-shot chain of thought, greedy, strict answer match — the protocol the Ouro paper specifies in its Table 16. The published figure for this model is 78.92.

Verification against the reference implementation

The port was validated against numbers measured with the original transformers implementation on the same machine before it existed, so this is a comparison to data rather than to an impression.

Every depth is within one standard error, greedy output is token-identical on a smoke prompt, and the model loads as n_layer = 96 at 1.43 B parameters — depth expanded, weights stored once. On an Apple M4 the GGUF runs about 3.4x faster than the reference does on MPS.

The loop count is a runtime dial

Unusually for a GGUF, the compute/accuracy trade-off is adjustable at load time from a single file, because llama.cpp's loader consults key overrides before the file:

llama-cli -m Ouro-1.4B-Q8_0.gguf --override-kv ouro.num_loops=int:1   # 24 layers, ~36 tok/s
llama-cli -m Ouro-1.4B-Q8_0.gguf --override-kv ouro.num_loops=int:2   # 48 layers
llama-cli -m Ouro-1.4B-Q8_0.gguf                                      # 96 layers, ~9.5 tok/s, default

Accuracy follows depth: 26 / 67 / 80 percent on GSM8K at 1 / 2 / 4 loops. Note the model was trained at 4 loops; the published ablation shows quality degrading beyond that.

Quantization notes

Q4KM costs about 5 points, more than a dense model this size usually loses. A natural hypothesis is that a looped model re-applies the same weight error once per loop, so the damage compounds with depth. That was tested and is false: the Q4 penalty is 6.0 points at depth 1 and 5.5 at depth 4, a difference of −0.5 against a combined standard error of 7.9. The cost is flat in depth. Q8_0 is effectively lossless and is the recommended file.

Always quote the loop depth alongside a number from these files — the same file scores 20% or 75% depending only on a load-time flag.

Provenance

Converted from ByteDance/Ouro-1.4B at commit 574fa66cb8bf5abdc979642d01cf2b79b16bfab1 with a llama.cpp built from upstream 67672dc plus the ouro architecture patch. The early-exit gate is deliberately not converted: it selects which already-computed loop feeds the LM head rather than changing what is computed, and at the shipped early_exit_threshold = 1.0 it never fires (measured: 0 of 299 token positions exit early).

Citation

@article{ouro2025,
  title  = {Scaling Latent Reasoning via Looped Language Models},
  author = {ByteDance Seed},
  journal= {arXiv:2510.25741},
  year   = {2025}
}