sixstringzen/Hemmingway-1-oQ4e-mtp

Hemmingway-1 oQ4e with MTP

This repository contains an enhanced oQ4e quantization of Altworld/Hemmingway-1 for MLX and oMLX on Apple silicon. The conversion preserves the model's multi-token prediction (MTP) tensors.

Altworld developed and published the source model. sixstringzen performed this conversion and published the converted weights with their quantization report. The original model, its intended use, and its training details remain documented in the source model card.

Quantization set

This repository is part of the Hemmingway-1 oMLX oQe Quantizations collection. Every build in the set uses the same source revision, group size, non-quantized dtype, calibration pass, and MTP preservation policy.

Quantization details

oQe uses activation importance to assign additional precision to sensitive tensors. This build starts with 4-bit weights, assigns 5 bits to 115 tensors, and stores language_model.lm_head at 8-bit because that tensor had no matching imatrix entry. The quantization report records no matrix-shape mismatches and no missing weight shards.

The included `oq_imatrix_report.json` records the sensitivity pass, calibration settings, tensor coverage, and fallback. Strict imatrix coverage was disabled for the known language_model.lm_head fallback.

Compatibility

This model was created and tested with oMLX 0.7.0.dev2. The source model identifies its text architecture as qwen3_5_text; the converted artifact uses qwen3_5, which matches the architecture name supported by this oMLX build.

The weights use MLX safetensors and are not GGUF files. Compatibility with other MLX runtimes or earlier oMLX releases has not been verified.

Use with oMLX

Download sixstringzen/Hemmingway-1-oQ4e-mtp from the oMLX model browser, then load it as an LLM. After the model is loaded, the following request uses the local OpenAI-compatible endpoint:

curl -s http://127.0.0.1:31423/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Hemmingway-1-oQ4e-mtp",
    "messages": [
      {
        "role": "user",
        "content": "Write one vivid sentence about rain on a city window."
      }
    ],
    "max_tokens": 80,
    "temperature": 0.7,
    "enable_thinking": false,
    "stream": false
  }'

Set enable_thinking to false when you want direct prose without visible planning. Runtime defaults and the registered model identifier can vary with the local oMLX installation.

Verification

The finished artifact passed a local oMLX smoke test on 2026-09-20. oMLX loaded the model, returned a complete chat response with finish_reason: stop, and unloaded it without error.

The artifact contains four safetensors shards, 1,876 indexed tensors, and 29 MTP tensors. The index references no missing shards. This smoke test confirms that the files load and generate through oMLX; it does not establish quality parity with the BF16 source model.

Quality evaluation (v1)

This model is part of the Hemmingway-1 oMLX quantization collection and was compared with a clean BF16 reference in a bounded blind A/B writing study.

Local measurement provenance

Generations were produced by real MLX/oMLX software on an Apple M5 Max with 128 GB unified memory under a deterministic quant-isolation profile. MTP and mixed-runtime speculative accelerators were disabled for the baseline. Captured local manifests, generation records, and oMLX/macmon telemetry are the measurement evidence. Claude monitored or orchestrated a subset of the local tests; Claude is workflow provenance, not the inference engine or measurement source.

Blind-judge result

The study used 14 writing tasks, three judge lanes, and normal/swapped response order. Results below pool the candidate comparisons across Claude Opus 5, Gemini 3.8 Flash, and Grok 4.7:

The hosted reference row in the local report is a cap-matched subset of 11 prompts, not another quantization condition. Raw inter-rater agreement was 83.7% across 606 pairings. Normal-versus-swapped agreement was 39.6% for Claude, 48.5% for Gemini, and 42.6% for Grok, so the results should be read as subjective and order-sensitive.

Interpretation for this build

Lower aggregate quality than oQ6e and oQ8e in this pass; not a quality-first choice from this study.

This pass does not establish mathematical equivalence, token-level fidelity, or general benchmark superiority. KLD, top-1/top-k agreement, KV-cache divergence, and broader benchmark task scores remain separate pending measurements. The public task and metadata package is the companion benchmark dataset.

Limitations

Quantization can change word choice, coherence, and instruction following. A controlled BF16 comparison has not been published for this build.

The sensitivity pass used oMLX's oqe_code_multilingual calibration dataset. No prose-specific calibration dataset was used. The MTP tensors are present in the artifact, but MTP-assisted decoding has not been benchmarked separately.

The original model's documented limitations and acceptable-use guidance also apply to this quantized release.

License

The source model is released under the Apache 2.0 license. This quantized derivative uses the same license; refer to the source repository for the upstream model card and attribution.

Feedback

Send compatibility reports through this repository's Community tab and include your oMLX version and Apple hardware.