Modelpublic

nightmedia/Gemma-3-27b-it-HERETIC-Gemini-1000x-Deep-Reasoning-q6-mlx

sourceHugging Faceupdated 8mo agoView on Hugging Face

0likes34downloads

Model Card

Gemma-3-27b-it-HERETIC-Gemini-1000x-Deep-Reasoning-q6-mlx

Quantized model performance

brainwave

q6      0.599,0.757,0.876,0.770,0.466,0.806,0.756

Brainwaves for regular vs 1000x models in the q6 quant

brainwave

regular  0.594,0.746,0.881,0.779,0.464,0.816,0.751
1000x    0.599,0.757,0.876,0.770,0.466,0.806,0.756

Heretic ablation improved the model arc/arc_easy significantly, with minor drops in other places

-G

Use with mlx

bash

pip install mlx-lm

python

from mlx_lm import load, generate

model, tokenizer = load("Gemma-3-27b-it-HERETIC-Gemini-1000x-Deep-Reasoning-q6-mlx")

prompt = "hello"

if tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, add_generation_prompt=True, return_dict=False,
    )

response = generate(model, tokenizer, prompt=prompt, verbose=True)