LuRenJiasWorld/RWKV-v7-0.1B-G1-GGUF

RWKV-v7-0.1B-G1-GGUF

GGUF-quantized RWKV v7 0.1B model files for use with **rwkz** — an LLM-powered lossless text compressor.

These are the same weights as BlinkDL/rwkv7-g1, converted to GGUF format with llama.cpp tooling. No fine-tuning or modification — just quantization for smaller download and faster load.

Intended Use: Lossless Text Compression

This model powers rwkz, a reimplementation of Fabrice Bellard's ts_zip. The RWKV v7 model predicts the next token in a text, and an arithmetic coder uses those predictions to compress the actual tokens. The better the model predicts, the fewer bits it takes.

Text → Tokenizer → RWKV v7 (predict) → Arithmetic Coder → .rkz compressed file

These are not chat / instruct models. They are base RWKV v7 language models, suitable only for next-token prediction. For chat, use BlinkDL/rwkv7-g1 directly.

Available Quantization Levels

All generated from the same F16 source using llama.cpp's llama-quantize. Weights are dequantized to F32 at load time for fast CPU inference.

Which Level Should I Use?

bpb (lower = better compression)
4.2 ┤ Q2_K ●  (96 MB)
3.6 ┤
3.4 ┤ Q3_K ●  (110 MB)   ← 15% smaller model, 13% worse bpb than 4-bit
3.2 ┤ Q4_1 ●  (135 MB)
     │ Q4_0 ●  (127 MB)
3.0 ┤ ● ● ● ● ● ●          ← Q4_K_M through F16: all ~3.01–3.06 bpb
     └────────────────────────────────────
         96   110  127  143  203  369 MB

Q4_K_M is the sweet spot — same size as Q4_0 (127 MB) but achieves 3.02 bpb vs 3.10. Going to higher bit-widths buys almost nothing. Below 4-bit, compression quality degrades noticeably.

K-Quant vs Legacy

At comparable sizes, K-quant types consistently outperform legacy types:

Always prefer K-quant when available.

Compression Performance vs Traditional Tools

Tested on alice29.txt (152 KB, English prose, i7-7700K CPU).

20 KB sample

rwkz achieves 16% better compression than bzip2 at 20 KB — at the cost of being ~30,000× slower.

Full file (152 KB, traditional compressors only)

rwkz hasn't been run on the full 152 KB file (estimated ~51 min on this CPU). With larger RWKV models (1.5B+), bpb is expected to drop below bzip2.

Reproducing Quantization

This repository includes everything needed to reproduce all quantization levels from the F16 source:

# Download the generation script
wget https://huggingface.co/LuRenJiasWorld/RWKV-v7-0.1B-G1-GGUF/resolve/main/scripts/generate_quants.sh

# Run with llama.cpp tools in PATH
LLAMA_CPP_BIN_DIR=/path/to/llama.cpp/build/bin ./generate_quants.sh

What's included:

• —`imatrix/rwkv7-0.1b-g1.imatrix` — Pre-computed importance matrix (WikiText calibration, ~500KB)
• —`calibration/calibration.txt` — Calibration data used (WikiText excerpt)
• —`scripts/generate_quants.sh` — One-shot script to regenerate all quantization levels

Model Details

Using with rwkz

# Compress (model auto-downloads if not cached locally)
rwkz compress input.txt output.rkz --q Q4_K_M

# Decompress (fingerprint-matched automatically)
rwkz decompress output.rkz restored.txt

rwkz automatically selects the best available quantization level. Pass --model to use a specific GGUF file directly.

License

Model weights: Apache 2.0 (same as BlinkDL/rwkv7-g1).

The rwkz compressor: Apache 2.0 License.

Acknowledgments

• —BlinkDL — RWKV v7 model architecture and training
• —Fabrice Bellard — Original ts_zip / rwkz concept
• —llama.cpp — GGUF format, quantization tools
• —bartowski — GGUF distribution best practices
• —ikawrakow — Calibration datasets