ShayonSarker/xglm-564M-GGUF

<div align="center">

<p> <a href="https://huggingface.co/ShayonSarker/xglm-564M-GGUF"><img alt="Hugging Face GGUF" src="https://img.shields.io/badge/Hugging%20Face-GGUF-FFD21E?style=for-the-badge"></a> <img alt="XGLM model" src="https://img.shields.io/badge/Model-XGLM--564M-8A2BE2?style=for-the-badge"> <img alt="GGUF formats" src="https://img.shields.io/badge/GGUF-F16%20%7C%20Q80%20%7C%20Q4K_M-FFD21E?style=for-the-badge"> <img alt="Languages" src="https://img.shields.io/badge/Languages-30%2B-00A6A6?style=for-the-badge"> <img alt="Bengali and Bangla" src="https://img.shields.io/badge/Bengali-Bangla-16A34A?style=for-the-badge"> <img alt="Runtime" src="https://img.shields.io/badge/runtime-llama.cpp%20%2B%20patch-E8590C?style=for-the-badge"> <img alt="Validated" src="https://img.shields.io/badge/validated-pass-22C55E?style=for-the-badge"> <img alt="F16 parity with transformers" src="https://img.shields.io/badge/F16%20parity-14%2F14%20exact-00A6A6?style=for-the-badge"> <img alt="MIT license" src="https://img.shields.io/badge/License-MIT-7C3AED?style=for-the-badge"> </p>

🌍 XGLM-564M GGUF

A compact multilingual base model for Bengali, English, and global language workloads.

🌐 30+ Languages &nbsp;•&nbsp; 🇧🇩 Bengali / Bangla &nbsp;•&nbsp; 🧠 Base Model &nbsp;•&nbsp; ⚙️ GGUF (llama.cpp + patch) &nbsp;•&nbsp; 📦 564M Parameters &nbsp;•&nbsp; ⚖️ MIT

Source model · llama.cpp

👇 View verified English and Bangla question/answer examples

</div>

✨ Highlights

• —XGLM support for llama.cpp via the included xglm-llama.cpp.patch (upstream does not ship this architecture)
• —256,008-token vocabulary with verified multilingual token parity
• —2,048-token context window
• —F16, Q80, and importance-matrix-calibrated Q4K_M formats

📦 Choose a Format

⚠️ Runtime requirement

Stock llama.cpp cannot load these files and fails with unknown model architecture: 'xglm'. XGLM is not in upstream llama.cpp. Apply the bundled patch against the pinned commit:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git checkout 6b790a9c291b5d7af3312bbf9f0c558aa023b13e
git apply /path/to/xglm-llama.cpp.patch
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release --target llama-completion

Alternatives that need no patch: transformers with facebook/xglm-564M, or the HF gguf file with a runtime that implements xglm.

🏗️ Rebuild

The GitHub release hub includes the end-to-end builder, XGLM runtime patch, verifier, and pinned dependencies.

git clone https://github.com/Dadhichi-Sarker-Shayon/XGLM-564M-GGUF.git
cd XGLM-564M-GGUF
python -m pip install -r requirements-build.txt
python build_gguf.py

The build requires at least 8 GB of free disk space. It does not overwrite this release.

🚀 Run Locally

Apply the runtime patch first, then:

hf download ShayonSarker/xglm-564M-GGUF XGLM-564M-Q4_K_M.gguf --local-dir .

llama-completion -m ./XGLM-564M-Q4_K_M.gguf \
  -p "Question: What is the capital of Japan?
Answer:" \
  -n 24 --temp 0

Expected output: The capital of Japan is Tokyo.

<a id="verified-question-answer-examples"></a>

❓ Verified Question → Answer Examples

Every row below is a verbatim XGLM-564M-Q4_K_M.gguf completion produced with the patched build at --temp 0, 24 new tokens, prompt form Question: ...\nAnswer:. Nothing is hand-written; rows that came out wrong are moved to the failure table below instead of being edited into looking correct. This is a smoke test, not a benchmark.

🔍 Is a wrong answer our bug or the model's? We tested it

14 prompts (8 English Q&A, 3 plain completions, 3 Bengali) were run through facebook/xglm-564M in transformers and through our GGUFs — all greedy, 24 new tokens, same prompt strings.

The conversion is faithful. F16 reproduces the reference model string-for-string on all 14 prompts, including the answers that are factually wrong — for example the reference answers "There are approximately 4,000 continents in the world" and our F16 file gives the identical sentence. A broken conversion or runtime patch could not track the reference this closely.

So the 564M model's factual failures below are the upstream checkpoint's own behaviour, not an artefact of this release.

⚠️ Why Q4KM looks worse than F16

Use F16 or Q8_0 when answer fidelity matters. At 564M parameters the model is already near its knowledge floor, so Q4KM tips it from "wrong but on-topic" into confabulation.

Perplexity does not predict this. The table below shows Q4KM English PPL within +4.4% of F16, yet it disagrees with the reference on 9 of 14 prompts. PPL is a text-fluency proxy, not an accuracy metric.

⚠️ Known Failures (Q4KM, the published default)

These are the exact completions the released Q4KM file produces:

Bengali/Bangla prompting on this checkpoint is unreliable in the reference model too. Use the 2.9B release for Bangla work, and expect factual errors from either model outside simple lookups.

📈 Performance

Lower perplexity (PPL) is better. Scores use separate held-out English and Bengali text with 64-token evaluation windows.

🔬 Validation

• —Generation parity with `transformers`: 14 greedy prompts (8 English Q&A, 3 plain completions, 3 Bengali) — F16 matches the reference output exactly on 14/14. Q4KM matches on 5/14. See the parity section above.
• —Token IDs match Transformers across Bengali, English, French, Chinese, and Arabic.
• —Both quantized formats pass the English and Bengali held-out perplexity gates.
• —Not tested: logit-level numeric parity, and long-context behaviour beyond 24 generated tokens.

🧩 Intended Use

XGLM-564M is a base language model, not an instruction-tuned assistant. It is suitable for compact multilingual research, Bengali/English experiments, local generation, and GGUF runtime testing.

Outputs may be inaccurate or inappropriate. Validate important results independently.

📄 License

MIT. See the upstream model card for source-model details and attribution.