Wondernutts/MS3.1-24B-Magnum-Diamond-int4-ov

MS3.1 24B Magnum Diamond — merged OpenVINO INT4

This repository contains the fully merged, text-only OpenVINO INT4 build of DS-Archive/MS3.1-24B-Magnum-Diamond-LoRA on its exact ZeroAgency text-only Mistral Small 3.1 base.

The Magnum Diamond adapter was created by Doctor Shotgun/DS-Archive. ZeroAgency provided the Hugging Face text-only base conversion. Wondernutts performed the validated BF16 merge and OpenVINO conversion to INT4. Read the adapter's original card for training details, intended use, and original credits.

What it is

The source is an rsLoRA creative-writing and roleplay tune intended to bring the prose character of the Magnum/Claude-inspired model family to Mistral Small 3.1 24B. The adapter card says it is designed to work with or without prepended character names and with or without a response prefill.

This repository is not an adapter. It contains the merged weights compressed into a self-contained OpenVINO model. You do not need PEFT or the source base at inference time.

The source adapter card notes that a newer MS3.2 adapter exists. This artifact intentionally preserves the named MS3.1 adapter and should not be mistaken for the newer release.

Conversion details

The exported config advertises 131,072 positions. The adapter was trained with a 32,768-token sequence length; behavior beyond that training window has not been independently validated for this conversion.

Download

Download the public repository:

python -m pip install "openvino-genai==2026.2.1.0" "huggingface_hub>=1.0"
hf download Wondernutts/MS3.1-24B-Magnum-Diamond-int4-ov --local-dir ./ms31-24b-magnum-diamond-int4-ov

Run on Intel GPU

import openvino_genai as ov_genai

model_dir = "./ms31-24b-magnum-diamond-int4-ov"
pipe = ov_genai.LLMPipeline(
    model_dir,
    "GPU",
    CACHE_DIR="./ov_cache",
    DYNAMIC_QUANTIZATION_GROUP_SIZE=128,
)

config = ov_genai.GenerationConfig()
config.max_new_tokens = 768
config.do_sample = True
config.temperature = 0.9
config.top_p = 0.95
config.repetition_penalty = 1.1

pipe.start_chat("You are a perceptive fantasy character. Stay in character and write vivid prose.")
print(pipe.generate("The campfire dies as footsteps circle beyond the trees.", generation_config=config))
pipe.finish_chat()

Use LLMPipeline; the base used here contains only the text component and does not support vision. start_chat() applies the bundled Mistral template and keeps conversation state in KV cache. Change "GPU" to "CPU" for CPU inference.

Arc Pro B70 benchmark

Tested on one Intel Arc Pro B70 with prefix caching disabled. Each PP result is the mean of two exact-token runs after one same-shape warmup. Decode used 128 forced output tokens.

DQGS=128 improved PP by 18.2% at 512, 37.8% at 2K, and 39.4% at 6K. The generated-token hashes matched DQGS=0 at every measured shape, including the 128-token decode run.

Intended use and limitations

The source adapter is intended for fiction, creative writing, and roleplay—not factual advice. Its biases and failure modes carry into this conversion, and INT4 compression may introduce additional quality loss. Treat generated material as fiction and evaluate it for your own application.

License and attribution

Apache-2.0, following the adapter and base repositories. Adapter/training credit: Doctor Shotgun / DS-Archive. Text-only base conversion: ZeroAgency. OpenVINO merge and conversion: Wondernutts.