dharun2049/Brahmaputra-1-GGUF-Q3_K_M

Brahmaputra-1 GGUF Q3KM

Brahmaputra-1 GGUF Q3_K_M is a quantized GGUF release of `dharun2049/Brahmaputra-1`.

Brahmaputra-1 is an experimental compact language model derived from `Qwen/Qwen2-0.5B-Instruct`.

The original Brahmaputra-1 checkpoint achieved an MMLU score of 42.95% in the evaluation used during development.

This quantization

Maximum compression among these releases

Useful for low-memory systems and lightweight local deployment.

Recommendation: Choose this when memory footprint matters more than maximum fidelity.

Running with llama.cpp

./llama-cli \
  -m Brahmaputra-1-Q3_K_M.gguf \
  -ngl 999 \
  -p "Explain reinforcement learning in simple terms." \
  -n 256

For CPU-only inference:

./llama-cli \
  -m Brahmaputra-1-Q3_K_M.gguf \
  -p "Write a Python function that checks whether a number is prime." \
  -n 256

Quantization variants

Brahmaputra-1 is published in multiple independent GGUF repositories:

Repositories follow this naming scheme:

dharun2049/Brahmaputra-1-GGUF-Q8_0
dharun2049/Brahmaputra-1-GGUF-Q6_K
dharun2049/Brahmaputra-1-GGUF-Q5_K_M
dharun2049/Brahmaputra-1-GGUF-Q4_K_M
dharun2049/Brahmaputra-1-GGUF-Q3_K_M

About Brahmaputra-1

Brahmaputra-1 explores how much useful capability can be retained in a very small language model through post-training and distillation.

The model is intended for experimentation with:

• —local inference
• —lightweight language-model deployment
• —model compression research
• —distillation research
• —edge and client-side AI experiments

Benchmark note

The reported 42.95% MMLU score belongs to the original Brahmaputra-1 checkpoint.

Quantization can slightly alter benchmark performance. Each GGUF variant should therefore be benchmarked independently before making quant-specific performance claims.

Original model

`dharun2049/Brahmaputra-1`

Base model

`Qwen/Qwen2-0.5B-Instruct`

License

Apache 2.0, following the licensing metadata of the source checkpoint.