jglowa/Bielik-Minitron-7B-v3.0-DFlash-GGUF

Bielik-Minitron-7B-v3.0-DFlash-GGUF

This repository provides GGUF files with F16 and Q8_0 quantization of Bielik-Minitron-7B-v3.0-DFlash model by SpeakLeash for use in llama.cpp.

Bielik-Minitron-7B-v3.0-DFlash is a DFlash draft model designed for use with Bielik-Minitron-7B-v3.0-Instruct. Its development and training were supported by computational grant PLG/2026/019559 and carried out on the Helios supercomputer, providing access to the advanced infrastructure and computational resources required for large-scale machine learning workloads.

This is not a standalone language model. It is designed to be paired with the target model in a speculative decoding setup.

How to use

Install llama.cpp and one of the GGUF files in the root directory, then run Bielik Minitron like this:

llama-cli -m Bielik-Minitron-7B-v3.0-Instruct-Q8_0.gguf --spec-type draft-dflash --spec-draft-model Bielik-Minitron-7B-v3.0-DFlash-Q8_0.gguf --spec-draft-ngl all --spec-draft-n-min 16 -fa on