CoolFace
Modelpublic

mishig/xet-gguf-edit-test

sourceHugging Faceupdated 1y agoView on Hugging Face
0likes1.5kdownloads
Model Card

GGUF Header Edit Benchmark

Benchmark script for measuring how long it takes to edit GGUF headers in-place on Hugging Face with streaming blobs (xet) and create a pull request per file. It fetches metadata, rebuilds the header with a small change, commits an edit (header slice only), and records timings to a CSV.

Result from benchmark.ts

<div class="flex">

<div style="max-width: 600px"> <img src="https://huggingface.co/mishig/xet-gguf-edit-test/resolve/main/output.png">

[!note] Rule of thumb (linear fit): timeminutes ≈ 0.36 × sizeGB + 0.25

</div>

Model Size (GB)Time (minutes)
0.50.28
1.00.47
1.50.24
2.01.06
2.51.29
3.01.43
3.51.59
4.01.61
4.51.82
5.01.98
5.52.10
6.02.18
6.52.14
7.04.73
7.55.04
8.02.71
8.52.75
9.03.03
9.53.11
10.03.24

</div>

✨ What this does

For each *.gguf file in a model repo:

  1. 1.Discover files via the Hugging Face model tree API.
  2. 2.Fetch GGUF + typed metadata with @huggingface/gguf.
  3. 3.Rebuild the header using buildGgufHeader (preserving endianness, alignment, and tensor info range).
  4. 4.Commit a slice edit (header bytes only) using commitIter with useXet: true to avoid full re-uploads.
  5. 5.Create a PR titled benchmark.
  6. 6.Record timing (wall-clock) to benchmark-results.csv.

🧱 Requirements

  • —Node 18+
  • —A Hugging Face token with read + write on the target repo: HF_TOKEN
  • —NPM packages:
  • —@huggingface/gguf
  • —@huggingface/hub
  • —Network access to huggingface.co

🔧 Setup

bash
npm i
npm run benchmark