CoolFace
Modelpublic

laion/marin-32b-base-GGUF

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face
0likes118downloads
Model Card

marin-32b-base-GGUF

GGUF quantizations of `marin-community/marin-32b-base`, a 32B dense base model (Qwen3 architecture).

These were produced with llama.cpp (convert_hf_to_gguf.py → f16 GGUF → llama-quantize).

Available quantizations

FileQuantNotes
marin-32b-base-Q4_K_M.ggufQ4KM4-bit, medium — good size/quality balance (recommended default)
marin-32b-base-Q5_K_M.ggufQ5KM5-bit, medium — higher quality, larger
marin-32b-base-Q6_K.ggufQ6_K6-bit — near-lossless
marin-32b-base-Q8_0.ggufQ8_08-bit — effectively lossless vs f16

Source

  • —Base model: marin-community/marin-32b-base (Apache-2.0)
  • —Architecture: Qwen3 (Qwen3ForCausalLM), 64 layers, hidden 5120, vocab 128256
  • —This is a base (non-instruct) model; there is no chat template.

Usage (llama.cpp)

bash
./llama-cli -m marin-32b-base-Q4_K_M.gguf -p "Your prompt here"