CoolFace
Modelpublic

Pluto-AI-Labs/Pluto-Genesis-0.6B-GGUF

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face
1likes248downloads
Model Card

Pluto-Genesis-0.6B-GGUF

GGUF quantizations of Pluto-Genesis-0.6B, an instruction-tuned language model built on Qwen3-0.6B.

🤗 Original Transformers Model https://huggingface.co/Siddh07ETH/Pluto-Genesis-0.6B The original repository contains the FP16 model, training details, benchmarks, tokenizer, and Transformers checkpoints.

Model Links

RepositoryDescription
Original Modelhttps://huggingface.co/Siddh07ETH/Pluto-Genesis-0.6B
GGUF Repositoryhttps://huggingface.co/Siddh07ETH/Pluto-Genesis-0.6B-GGUF

Overview

Pluto-Genesis-0.6B-GGUF provides optimized GGUF quantizations of the Pluto-Genesis-0.6B language model for efficient local inference.

These models are fully compatible with modern GGUF runtimes including:

  • —llama.cpp
  • —Ollama
  • —LM Studio
  • —Jan
  • —KoboldCpp
  • —text-generation-webui
  • —GPT4All
  • —Any GGUF-compatible inference engine

Available Quantizations

FileQuantizationSizeRecommended Use
Pluto-Genesis-0.6B-F16.ggufF161.20 GBMaximum quality, research and archival
Pluto-Genesis-0.6B-Q8_0.ggufQ8_00.64 GBHighest quality local inference
Pluto-Genesis-0.6B-Q6_K.ggufQ6_K0.50 GBExcellent balance of quality and speed
Pluto-Genesis-0.6B-Q5_K_M.ggufQ5KM0.44 GBRecommended for most users ⭐
Pluto-Genesis-0.6B-Q4_K_M.ggufQ4KM0.40 GBLowest memory usage and fastest inference

Quantization Guide

QuantizationQualitySpeedMemory
F16⭐⭐⭐⭐⭐⭐⭐Highest
Q8_0⭐⭐⭐⭐☆⭐⭐⭐High
Q6_K⭐⭐⭐⭐☆⭐⭐⭐⭐Medium
Q5_K_M⭐⭐⭐⭐⭐⭐⭐⭐⭐Low
Q4_K_M⭐⭐⭐⭐⭐⭐⭐⭐Lowest
Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):

<p align="center"> <img src="./quant.png" width="50%"> </p>

Quick Start

llama.cpp

Download the recommended Q5KM model:

bash
huggingface-cli download Siddh07ETH/Pluto-Genesis-0.6B-GGUF \
Pluto-Genesis-0.6B-Q5_K_M.gguf \
--local-dir ./models

Run inference:

bash
llama-cli \
-m ./models/Pluto-Genesis-0.6B-Q5_K_M.gguf \
-p "<|im_start|>user
Explain quantum computing in simple terms.
<|im_end|>
<|im_start|>assistant" \
-n 256

Ollama

Create the model:

bash
ollama create pluto-genesis-0.6b -f Modelfile

Run:

bash
ollama run pluto-genesis-0.6b

Compatibility

These GGUF files have been generated for use with:

  • —llama.cpp
  • —Ollama
  • —LM Studio
  • —Jan
  • —KoboldCpp
  • —GPT4All
  • —text-generation-webui

Conversion Details

PropertyValue
Original Modelhttps://huggingface.co/Siddh07ETH/Pluto-Genesis-0.6B
GGUF Repositoryhttps://huggingface.co/Siddh07ETH/Pluto-Genesis-0.6B-GGUF
Base ModelQwen/Qwen3-0.6B
FormatGGUF
Conversion Toolllama.cpp (convert_hf_to_gguf.py)
llama.cpp Versionb5604
Converted2026-07-12T12:15:48Z

File Integrity

SHA-256 hashes for every GGUF file are available in:

CHECKSUMS.txt

Verify downloaded files before deployment for maximum integrity.

License

This project is distributed under the Apache License 2.0.

Please also review the license of the original base model:

  • —Qwen/Qwen3-0.6B

📄 Research Paper

The methodology and benchmark evaluation for Pluto-Genesis are described in the accompanying research paper.

DOI: https://doi.org/10.5281/zenodo.21368749

Citation

If you use Pluto-Genesis in your research or applications, please cite:

@misc{pluto-genesis,
  title={Pluto-Genesis-0.6B},
  author={Siddh07ETH},
  year={2026},
  publisher={Hugging Face},
  howpublished={https://huggingface.co/Siddh07ETH/Pluto-Genesis-0.6B}
}

Author

Siddh07ETH

  • —🤗 Original Model: https://huggingface.co/Siddh07ETH/Pluto-Genesis-0.6B
  • —🤗 GGUF Repository: https://huggingface.co/Siddh07ETH/Pluto-Genesis-0.6B-GGUF

Built with Qwen3-0.6B, llama.cpp, and the open-source AI ecosystem.