Abiray/Qwen3.5-4B-Python-Coder-GGUF

1---2license: apache-2.03base_model: Jackrong/Qwen3.5-4B-Python-Coder4language:5- en6pipeline_tag: text-generation7tags:8- gguf9- qwen10- qwen3.511- code12- python13---14 15# Qwen3.5-4B-Python-Coder-GGUF16 17## Available Quantizations18 19The following quantization formats are available in this repository:20 21* **Q3_K_M:** Smallest size, heavily quantized. Good for very low RAM environments, but significant loss in coding accuracy.22* **Q4_K_M:** Recommended baseline. Excellent balance between file size, memory usage, and coding performance.23* **Q5_K_M:** Higher accuracy than Q4, slightly larger file size. 24* **Q6_K:** Very close to the original unquantized model's performance. Great if you have the RAM for it.25* **Q8_0:** Almost zero quality loss compared to the original 16-bit model, but largest file size and highest memory requirement.26 27## How to Run28 29You can run these models locally using [llama.cpp](https://github.com/ggerganov/llama.cpp) or compatible interfaces like LM Studio, Ollama, or text-generation-webui.30 31**Example using `llama.cpp` in the terminal:**32 33```bash34./main -m Qwen3.5-4B-Python-Coder-Q4_K_M.gguf -n 512 --color -i -cml -p "<|im_start|>user\nWrite a Python script to scrape a website.<|im_end|>\n<|im_start|>assistant\n"