CoolFace
Modelpublic

DuoNeural/Ministral-8B-Instruct-GGUF

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face
1likes113downloads
Model Card

Ministral-8B-Instruct — GGUF Quants

Quantized GGUF versions of mistralai/Ministral-8B-Instruct-2410 — Mistral AI's Ministral 8B instruct model, optimized for edge and on-device deployment. Features sliding window attention for efficient long-context processing.

Available Files

FileQuantSizeUse Case
Ministral-8B-Instruct-Q8_0.ggufQ8_0~8.5GBMaximum quality
Ministral-8B-Instruct-Q6_K.ggufQ6_K~6.6GBNear-lossless
Ministral-8B-Instruct-Q5_K_M.ggufQ5KM~5.7GBHigh quality
Ministral-8B-Instruct-Q4_K_M.ggufQ4KM~4.9GBRecommended default
Ministral-8B-Instruct-Q3_K_M.ggufQ3KM~3.9GBLow VRAM
Ministral-8B-Instruct-IQ4_XS.ggufIQ4_XS~4.3GBImatrix 4-bit
Ministral-8B-Instruct-IQ3_XXS.ggufIQ3_XXS~3.2GBImatrix 3-bit
Ministral-8B-Instruct-IQ2_M.ggufIQ2_M~2.8GBImatrix 2-bit
Ministral-8B-Instruct-IQ1_S.ggufIQ1_S~2.0GBExtreme compression
Ministral-8B-Instruct-fp16.ggufFP16~16.0GBFull precision
imatrix.dat——Importance matrix

Usage

bash
./llama-cli -m Ministral-8B-Instruct-Q4_K_M.gguf \
  --ctx-size 8192 -n 512 \
  -p "[INST] Hello! [/INST]"

ollama run hf.co/DuoNeural/Ministral-8B-Instruct-GGUF:Q4_K_M
  • —Parameters: 8B | License: Apache 2.0 | Context: 32K (SWA)

Quantized by DuoNeural using llama.cpp on RTX 5090.

DuoNeural

DuoNeural is an open AI research lab — human + AI in collaboration.

PlatformLink
HuggingFacehuggingface.co/DuoNeural
Websiteduoneural.com
GitHubgithub.com/DuoNeural
X / Twitter@DuoNeural
Emailduoneural@proton.me
Newsletterduoneural.beehiiv.com
Supportbuymeacoffee.com/duoneural

DuoNeural Research Publications

TitleDOI
Nano-CTM: Ternary Continuous Thought Machines with Thought-Space Self-Prediction for Efficient Iterative Reasoning10.5281/zenodo.19775622
Recurrence as World Model: CTM Learns Implicit Belief States in Partially Observable Physical Environments10.5281/zenodo.19810620
Per-Object Slot Decomposition for Scalable Neural World Modeling: When Does Attention Beat Mean-Field?10.5281/zenodo.19846804
The Dynamical Horizon Principle: CTM Gates Converge to the Predictability Limit of Dynamical Systems10.5281/zenodo.19952612

Open access, CC BY 4.0. Authored by Archon, Jesse Caldwell, Aura — DuoNeural.