Modelpublic

nightmedia/granite-4.1-3B-TNG-Claude-Coder-xLAM-Heretic-q8-hi-mlx

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face

0likes91downloads

Model Card

granite-4.1-3B-TNG-Claude-Coder-xLAM-Heretic-q8-hi-mlx

This model is a NuSLERP merge of:

—treadon/granite-4.1-3b-Abliterated-AND-Disinhibited
—nightmedia/granite-4.1-3B-TNG-Coder-Heretic
—nightmedia/granite-4.1-3B-TNG-Claude-Coder-Heretic
—ermiaazarkhalili/Granite-4.1-3B-SFT-Claude-Opus-Reasoning-Unsloth
—ermiaazarkhalili/Granite-4.1-3B-Function-Calling-xLAM-Unsloth

brainwaves

quant    arc   arc/e boolq hswag obkqa piqa  wino
bf16     0.497,0.690,0.844
q8-hi    0.499,0.694,0.845,0.700,0.430,0.781,0.689
qx86-hi  0.491,0.685,0.844
mxfp8    0.467,0.662,0.814

Quant    Perplexity      Peak Memory   Tokens/sec
bf16     4.383 ± 0.029   9.74 GB       1743
q8-hi    4.381 ± 0.029   6.90 GB       1627
qx86-hi  4.398 ± 0.029   6.44 GB       1363
mxfp8    4.907 ± 0.033   6.58 GB       1474

Model components

granite-4.1-3B-TNG-Claude-Coder-Heretic

brainwaves

         arc   arc/e boolq hswag obkqa piqa  wino
q8-hi    0.496,0.683,0.828,0.692,0.422,0.775,0.676
         Perplexity      Peak Memory   Tokens/sec
q8-hi    4.447 ± 0.030   6.90 GB       1609

granite-4.1-3B-TNG-Coder-Heretic

brainwaves

         arc   arc/e boolq hswag obkqa piqa  wino
q8-hi    0.486,0.673,0.826,0.688,0.430,0.771,0.666
         Perplexity      Peak Memory   Tokens/sec
q8-hi    4.657 ± 0.032   6.90 GB       1626

ermiaazarkhalili/Granite-4.1-3B-SFT-Claude-Opus-Reasoning-Unsloth

brainwaves

         arc   arc/e boolq hswag obkqa piqa  wino
q8-hi    0.417,0.585,0.651,0.682,0.390,0.781,0.669

ermiaazarkhalili/Granite-4.1-3B-Function-Calling-xLAM-Unsloth

brainwaves

         arc   arc/e boolq hswag obkqa piqa  wino
q8-hi    0.423,0.602,0.850,0.641,0.428,0.743,0.619

Parent model

treadon/granite-4.1-3b-Abliterated-AND-Disinhibited

brainwaves

quant    arc   arc/e boolq hswag obkqa piqa  wino
mxfp8    0.405,0.598,0.843,0.520,0.442,0.713,0.582

Baseline model

ibm-granite/granite-4.1-3B

brainwaves

quant    arc   arc/e boolq hswag obkqa piqa  wino
mxfp8    0.406,0.581,0.821,0.484,0.434,0.712,0.559

The granite-4.1-3B-TNG-Coder-Heretic model was trained on the Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-qx86-hi-mlx with a custom Holodeck template that generated the characters.

This model is deeply immersed in Star Trek lore, with an intimate understanding of DS9 station operations and character strengths and weaknesses. It has not been explicitly trained on Star Trek, just on what the large model knows about Star Trek.

The traces have been generated using a custom system prompt, and contain general backend engineering questions in Haskell, Python, Golang, etc..

Additionally to the 544 TNG-flavored training, there are 986 questions from Polaris Alpha, but distilled from the MTP-Holo3-Qwopus.

The model can deliver output in a variety of formats depending on the type of question and complexity.

More details coming soon.

-G

Training parameters:

train

model: granite-4.1-3B-TNG-Coder-Heretic
source: treadon/granite-4.1-3b-Abliterated-AND-Disinhibited
dataset: nightmedia/chat_tng_qwen_polaris_544_986_1530.jsonl
Iter 550:
  Val loss       0.902
  Train loss     0.817
  Learning Rate  1.000e-05
  Tokens/sec     573.023
  Trained Tokens 804064
  Peak mem       28.964 GB

Model recipe

recipe

models:
  - model: granite-4.1-3B-TNG-Coder-Heretic
    parameters:
      weight: 1.6
  - model: ermiaazarkhalili/Granite-4.1-3B-SFT-Claude-Opus-Reasoning-Unsloth
    parameters:
      weight: 0.4
merge_method: nuslerp
dtype: bfloat16
name: granite-4.1-3B-TNG-Claude-Coder-Heretic

models:
  - model: granite-4.1-3B-TNG-Claude-Coder-Heretic
    parameters:
      weight: 1.6
  - model: ermiaazarkhalili/Granite-4.1-3B-Function-Calling-xLAM-Unsloth
    parameters:
      weight: 0.4
merge_method: nuslerp
dtype: bfloat16
name: granite-4.1-3B-TNG-Claude-Coder-xLAM-Heretic
name:

Use with mlx

bash

pip install mlx-lm

python

from mlx_lm import load, generate

model, tokenizer = load("granite-4.1-3B-TNG-Claude-Coder-xLAM-Heretic-q8-hi-mlx")

prompt = "hello"

if tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, add_generation_prompt=True, return_dict=False,
    )

response = generate(model, tokenizer, prompt=prompt, verbose=True)