Modelpublic

lemuralabs/Qwen3.5-122B-A10B-Abliterated-MLX-3

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

1likes144downloads

Model Card

Qwen3.5-122B-A10B-Abliterated-MLX-3

Qwen3.5-122B-A10B-Abliterated-MLX-3 is an Apple Silicon-oriented MLX quantization of the abliterated Qwen3.5 122B-A10B vision-language model.

<p>

</p>

Publisher: Lemura Labs Hugging Face organization: [Lemura Labs](https://huggingface.co/lemuralabs)

This release is the 3-bit / group-size-32 build selected from our vision-quantized candidate run. This model quantizes both the language side and eligible multimodal/vision modules.

Source And Credits

This quant was produced from the local full-precision abliterated model derived from:

—Abliterated source: Chompa1422/Qwen3.5-122B-A10B-abliterated
—Base model: Qwen/Qwen3.5-122B-A10B

Thank you to the Qwen team for the base model, to Chompa1422 for publishing the abliterated source model that made this quantization work possible, and to Pliny the Liberator for the broader abliterated-model research culture that inspired this release.

Quantization

Field	Value
Runtime format	MLX / MLX-VLM safetensors
Quantization method	MLZTQ `0.2` custom MLX-VLM affine quantization
Weight bits	3
Group size	32
Mode	affine
Quant profile	`mlztq-0.2-visionq_w3-g32`
Quant predicate	`visionq_w3`
Language weights	Quantized
Vision/multimodal weights	Quantized where MLX shape constraints allow
KV cache	TurboQuant runtime contract recorded; model weights are MLX affine quantized

Tensor spot checks from the selected artifact:

Tensor	dtype	Shape	Meaning
`language_model.model.layers.0.linear_attn.in_proj_qkv.weight`	`uint32`	`(12288, 288)`	Language side quantized
`language_model.model.layers.0.linear_attn.in_proj_qkv.scales`	`bfloat16`	`(12288, 96)`	Language affine scales
`vision_tower.blocks.0.attn.qkv.weight`	`uint32`	`(3456, 108)`	Vision side quantized
`vision_tower.blocks.0.attn.qkv.scales`	`bfloat16`	`(3456, 36)`	Vision affine scales

File Details

Item	Value
Safetensor shards	12
Indexed tensor payload	61,488,610,272 bytes
Local disk footprint	57 GiB
Decimal payload size	61.49 GB
Model index	`model.safetensors.index.json`
Quant manifest	`mlztq_manifest.json`
Processor files	`processor_config.json`, `preprocessor_config.json`, `video_preprocessor_config.json`
Tokenizer files	`tokenizer.json`, `tokenizer_config.json`, `vocab.json`, `chat_template.jinja`

Benchmarks

Benchmarks were run with the local MLZ deterministic VLM retention gate on 2026-05-02. Raw JSON, CSV, and Markdown benchmark artifacts are included under benchmarks/results/.

Bucket	Correct	Total	Accuracy
Quant health	4	4	100.0%
Text / reasoning	6	6	100.0%
OCR / document	6	6	100.0%
Vision general	5	10	50.0%
Overall gate	21	26	80.8%

Benchmark coverage:

Bucket	Sources
Quant health	Safetensor load, vision path availability, text canary, vision canary
Text / reasoning	MMLU-Pro, MMLU, GSM8K
OCR / document	ChartQA, DocVQA, local PDF pages with images/text
Vision general	RealWorldQA, AI2D, MMMU Accounting, MMMU Biology, MMMU Physics

Candidate comparison from the same run:

Candidate	Weight bits / group	Payload GB	Quant health	Text / reasoning	Vision general	OCR / document	Overall	Status
`visionq-w6g32`	6 / 32	107.40	100.0%	83.3%	50.0%	100.0%	76.9%	kept locally
`visionq-w5g32`	5 / 32	deleted	100.0%	100.0%	40.0%	83.3%	73.1%	pruned
`visionq-w4g32`	4 / 32	76.79	100.0%	83.3%	40.0%	100.0%	73.1%	kept locally
`visionq-w3g32`	3 / 32	61.49	100.0%	100.0%	50.0%	100.0%	80.8%	selected
`visionq-w2g32`	2 / 32	deleted	100.0%	16.7%	20.0%	50.0%	38.5%	pruned

Inference Engines And Apps

This is an MLX / MLX-VLM safetensors repository for Apple Silicon. It is not a GGUF, AWQ, GPTQ, EXL2, or Transformers fp16 repository.

Engine / app	Status for this repository	Notes
MLX-VLM	Intended reference path	Best target for image-text inference because this artifact keeps the VLM processor files and MLX-VLM tensor layout. Requires a loader/runtime version with Qwen3.5 MoE VLM support and affine quantized multimodal weights.
LM Studio on Apple Silicon	Intended app target, runtime-dependent	LM Studio's unified MLX engine uses `mlx-lm` for text generation and `mlx-vlm` for vision embeddings. Use a recent LM Studio build with MLX support; macOS 14+ is required for MLX models according to LM Studio's system requirements.
Custom MLX Python runtimes	Supported if they implement this architecture	Works for runtimes that can read MLX safetensors, Qwen3.5 MoE configs, the chat template, and MLX affine quantized language + vision tensors.
`mlx-lm` alone	Text-side only / not sufficient for full VLM use	`mlx-lm` is useful in the MLX ecosystem, but full image input needs the VLM path and processor stack.
Hugging Face Transformers / vLLM / TGI	Not directly loadable	These engines do not load this MLX quantized artifact directly. Use the original/full-precision model or produce a separate backend-specific quant.
Ollama / llama.cpp / KoboldCpp	Not directly loadable	These generally expect GGUF for local quantized inference. This repo is MLX safetensors, not GGUF.

Practical expectation: use this model on high-memory Apple Silicon Macs through MLX-VLM-compatible tooling. For image input, use PNG, JPEG, WebP, and PDF/image workflows supported by the serving app or preprocessing pipeline.

Research And Safety Notice

Why is this model Abliterated?

This model is intended for research, local experimentation, red-team evaluation, and authorized security testing. It may produce content that aligned models normally refuse. Users are responsible for applying appropriate safeguards and complying with laws and platform policies.

This is an abliterated model released for research and development, model-behavior analysis, authorized security testing, and experimentation with local Apple Silicon inference. Abliterated models may respond differently from aligned instruction models. Users are responsible for complying with applicable laws, platform policies, and safety requirements. The authors and uploaders are not responsible for misuse, harm, or unlawful deployment.

Reproducibility

The included mlztq_manifest.json records the source path, quantization recipe, weight format, vision quantization policy, and runtime contract used for this artifact. The benchmark files under benchmarks/results/ record the exact gate rows used to choose this model.