robertobissanti/EngGPT2-16B-A3B-GGUF

EngGPT2-16B-A3B GGUF

This repository contains a GGUF conversion of `engineering-group/EngGPT2-16B-A3B`.

The model inherits the original EngGPT Non-Commercial License and is intended for research and non-commercial use only.

Files

• —enggpt2-bf16.gguf: BF16 GGUF conversion.
• —enggpt2-repro-notes.md: conversion and runtime notes.
• —enggpt2-llama-cpp-support.patch: patch used for llama.cpp support.
• —enggpt2-config-for-conversion.json: local conversion config.

Runtime Requirement

This GGUF requires a patched llama.cpp runtime. An unpatched runtime may fail with:

wrong number of tensors; expected 291, got 243

The required runtime changes add support for optional attention Q/K RMSNorm tensors in the LLaMA loader and apply those norms before RoPE. The converter patch also maps EngGPT2 MoE expert tensor names into a Mixtral-compatible GGUF layout.

Build

For a static local build:

cmake -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF
cmake --build build --target llama-cli llama-server llama-completion -j 8

Quick Test

./build/bin/llama-cli \
  -m enggpt2-bf16.gguf \
  -p "Spiega in italiano cos'e un trasformatore elettrico." \
  -n 300 \
  --temp 0

Ollama

Stock Ollama does not run this GGUF unless its bundled llama.cpp runner includes the same runtime patch.

Related Repositories

• —Original model: `engineering-group/EngGPT2-16B-A3B`
• —llama.cpp fork: `robertobissanti/llama.cpp`
• —MLX workflow: `robertobissanti/enggpt2-mlx-manager`