oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-Llama

<p align="center"> <a href="https://nowpayments.io/donation?apikey=f1399916-9760-4d45-9129-b972124b8186" target="blank" rel="noreferrer noopener"> <img src="https://nowpayments.io/images/embeds/donation-button-white.svg" alt="Cryptocurrency & Bitcoin donation button by NOWPayments" /> </a> </p>

Qwen3.6-35B v2

[image]

👁️ Understand images. 💻 Build things. 🎭 Make it personal.

Portable GGUF weights for llama.cpp.

BF16 / FT · GGUF / Llama · Ollama · v2 collection

Jump to: Model · Setup · Reliable runtime · Hardware · Additional notes · License · Collection

<a id="meet"></a>

✨ Meet the model

Meet your local AI companion for ideas, code, images and conversation. Qwen3.6-35B v2 by oktayd brings these interests together in one downloadable model. Choose the edition that fits your setup, give it a task and shape its style with your own instructions.

<details> <summary>🧾 Checkpoint at a glance</summary>

Qwen3.6-35B-A3B family · qwen3_5_moe · about 35B total / 3B active parameters · 262,144 native context · BF16 master checkpoint with preserved vision and MTP tensors. The BF16 edition is the master checkpoint for server/FreeToken use; this page provides portable GGUF weights for llama.cpp. Native context is an architecture limit: the recommended runtime profile deliberately uses 131,072 tokens, while 9,600 is benchmark-only.

</details>

<details> <summary>✨ What you can do with it</summary>

These are things to try, not promises of perfect results. It can make mistakes or repeat itself; check important answers. Adult-oriented training is included (18+).

</details>

<details> <summary>🧠 A team of experts inside one model</summary>

MoE means Mixture of Experts. Think of a team of specialists: for each token, a router chooses which expert networks should contribute. This model has about 35 billion parameters in total, with roughly 3 billion active per token; its configuration selects 8 of 256 routed experts.

That saves computation compared with activating every expert at once. It does not turn a 35B download into a 3B-sized model: the expert weights still need disk space and accessible RAM/VRAM. Quantized editions make local use more practical. The experts are learned networks, not separate installed apps or named profession-specific agents.

</details>

<details> <summary>🕶️ Street knowledge. Business mind. Your style.</summary>

The personality direction is direct, sharp-witted and business-minded: a street-smart conversation partner with humor, creative confidence and room for disagreement. The training mix includes internet culture, slang, practical business topics and personality-oriented conversations. Think less formal textbook, more an opinionated partner for brainstorming, writing and exploring alternatives.

You set the tone: professional, casual, blunt or playful. The aim is personality without blind agreement. This is a style and training focus, not a guarantee of factual expertise or flawless judgment.

</details>

<a id="training-data"></a> <details> <summary>📚 Training data & attribution — 34,000 record uses across three runs</summary>

The three runs used 6,000 + 12,000 + 16,000 record uses. A record use is not necessarily a unique example; selected samples were used rather than entire upstream datasets. Training loss and record counts are not benchmark accuracy.

Private, manually prepared material accounts for 3,070 broad adult-learning uses and 133 synthetic 3D multi-view adult-learning uses; neither package is distributed. The public-source pool contains selected subsets from datasets in the categories above, including agents/search, coding, science, finance, security, visual QA and captions. Source URLs identify attribution only; their licenses are not replaced by this model’s license. Some retained local adapters have no recoverable upstream URL and are not represented as guessed sources.

</details>

<a id="start-on-your-device"></a>

🚀 Start on your device

Choose your hardware, then expand Ubuntu/Linux or Windows/PowerShell inside it. You only need one edition and one quantization.

<a id="reliable-runtime"></a>

One install → configured reliable runtime

This is the normal operating profile, not a benchmark override. Paste one command. It downloads IQ4_XS, writes the full llama.cpp and Ollama profiles, creates the launch/client scripts, embeds the personality and strict final-output system prompt, and updates a local Hermes configuration when present.

Prerequisite: have a CUDA-enabled llama-server.exe on PATH for the default direct launcher, or Ollama on PATH when selecting -Runtime Ollama. The installer handles all Q36 download and configuration after that.

irm https://huggingface.co/oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-Llama/resolve/main/tools/windows/Install-Q36-Reliability-Profile.ps1 | iex

Then launch one backend and use its configured client:

& C:\AI\Q36-Reliable\Start-Q36-Reliable.ps1
& C:\AI\Q36-Reliable\Invoke-Q36-Reliable.ps1 -Prompt "Hello Q36"

The default is llama.cpp. To create the same persistent profile as an Ollama model instead, run the same one-line installer with -Runtime Ollama (Ollama keeps an additional local model copy):

& ([scriptblock]::Create((irm https://huggingface.co/oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-Llama/resolve/main/tools/windows/Install-Q36-Reliability-Profile.ps1))) -Runtime Ollama

<details> <summary>⚙️ What the installer enforces & local regression evidence</summary>

The generated config/q36-reliability-profile.json is the single source of truth. It applies 131072 context, one request slot, CPU/RAM-resident weights, q8_0 K/V cache, Flash Attention, fitting with a 1536-MiB GPU reserve, continuous batching, finite 4096-token output, non-thinking sampling (0.7 / 0.8 / 20 / 0), and 1.08 / 128 repetition protection. The Ollama Modelfile receives the matching sampling, loop guard and system profile.

The embedded system profile preserves Q36's direct, sharp-witted, business-minded personality and includes this strict behavior for machine-sensitive results: “You are a strict final-output compiler. Follow every explicit output constraint. Before answering, silently verify exact text, required counts, format, and forbidden extras.” The supplied llama.cpp client wrapper applies it on every command and can retry invalid JSON or an exact expected string up to three times. Hermes is updated to apply it to every Q36 command. Ollama embeds it in the created model. Raw llama.cpp HTTP callers must include the generated system profile themselves: current llama-server has no server-global system-prompt flag.

The 9,600-token Q4KM configuration is benchmark-only and is deliberately not installed. See benchmark findings and normal-runtime fixes for reproducibility boundaries.

Local regression on 2026-09-06 used the same 8-case exact-output suite, 9,600-token context, Qwen non-thinking sampling, the strict message above, and bounded validator repair on an RTX 4060 Laptop 8 GB / 32 GB RAM system:

A second regression used 20 evenly spaced rows from the official 300-row IFBench test set. The IFBench verifier was not modified:

The optimized path validates every first answer, returns measured failures to the model, and allows at most three LLM correction attempts. A registry of narrow deterministic rule compilers runs only for exact mechanical constraints that do not converge reliably, including word positions/counts, indentation, HTML italics, sentence ratios, alphabet loops, palindromes, syllable parity, vowel limits, and forbidden whitespace. The final answer must then pass IFBench's unmodified strict verifier. --require-all makes any partial strict result fail the process and writes a machine-readable summary.json. The reproducible runner is included as `tools/run-ifbench.py`.

# Run from an official IFBench checkout against an OpenAI-compatible local endpoint.
uv run python /path/to/tools/run-ifbench.py --limit 20 --port 11442 \
  --model q36-laptop --context-tokens 9600 --max-tokens 1200 \
  --temperature 0.7 --top-p 0.8 --top-k 20 --min-p 0 \
  --repeat-penalty 1.08 --strict-system-prompt \
  --max-validation-retries 3 --deterministic-formatter --require-all \
  --output-root ./runs --ifbench-root .

These are local runtime reliability regressions, not raw full-dataset benchmark scores and not evidence that every arbitrary request will pass. In particular, 20/20 describes this fixed twenty-row regression sample, not all 300 IFBench rows. IFBench's verifier measures the stated structural constraints, not factual or semantic quality; those require separate checks. Narrow compilers can also make prose less natural and should be reserved for machine-sensitive output rather than ordinary chat. The 9,600-token setting here is benchmark-only; apply the same decoding, finite-output and validation rules at your regular chosen context. See Qwen's official Qwen3 sampling guidance, the official IFBench repository, and llama.cpp server options.

</details>

<a id="hardware-quickstarts"></a>

Hardware quickstarts

<details> <summary>🖥️ H200 / large server — full BF16 with Transformers</summary>

For a machine with enough GPU memory. Validated on Ubuntu / NVIDIA H200 with Transformers 5.16.1. BF16 weights alone are about 70 GB, plus runtime memory. A Windows client does not provide the remote server's VRAM; this package does not fit a single RTX 5090 or an 8-GB laptop GPU.

<details> <summary>🐧 Ubuntu / Linux — installation</summary>

Install Python 3.11, virtual-environment support and a CUDA-enabled PyTorch build from the official PyTorch selector that matches your NVIDIA driver. On a fresh Ubuntu machine:

sudo apt-get update
sudo apt-get install -y python3-venv python3-pip
mkdir -p qwen-v2-server
cd qwen-v2-server
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
# Install the CUDA-enabled PyTorch command from the official selector here.
python -m pip install transformers==5.16.1 accelerate pillow
python -c "import torch; print(torch.cuda.is_available()); print(torch.cuda.get_device_name(0))"

If the GPU check fails, fix the driver/PyTorch installation before downloading the model. Save the shared Python example below as qwen_chat.py, then run python qwen_chat.py. The first run downloads the BF16 weights.

</details>

<details> <summary>🪟 Windows / PowerShell — native installation</summary>

Native Windows — WSL2 is not required. Install Python 3.11 and a compatible NVIDIA driver, then use PowerShell 7. This native installation route is separate from the measured H200 validation, which was performed on Ubuntu:

New-Item -ItemType Directory -Path .\qwen-v2-server -Force | Out-Null
Set-Location .\qwen-v2-server
py -3.11 -m venv .venv
.\.venv\Scripts\python.exe -m pip install --upgrade pip
# Run the Windows CUDA PyTorch install command from the official selector,
# using .\.venv\Scripts\python.exe -m pip instead of pip.
.\.venv\Scripts\python.exe -m pip install transformers==5.16.1 accelerate pillow
.\.venv\Scripts\python.exe -c "import torch; print(torch.cuda.is_available()); print(torch.cuda.get_device_name(0))"

Save the shared example below as qwen_chat.py, then execute .\.venv\Scripts\python.exe .\qwen_chat.py. Use only Windows GPU/driver combinations supported by PyTorch and with sufficient memory.

If the H200 belongs to a remote Ubuntu server, Windows is just the client. Connect using Windows OpenSSH (replace the example and use your actual port/key), then follow the Ubuntu instructions in that remote shell:

ssh user@YOUR_SERVER

<details> <summary>🐧 Optional: Ubuntu under WSL2</summary>

Optional alternative, not required for native Windows. Follow Microsoft's WSL installation guide. From an administrator PowerShell, only if you want this alternative:

wsl --install -d Ubuntu

Restart if requested and complete Ubuntu's first-run account setup. Then open Ubuntu:

wsl -d Ubuntu

Inside Ubuntu, follow this device's Ubuntu/Linux instructions and create a separate Linux Python environment. Keep the Windows NVIDIA driver current; use NVIDIA's CUDA-on-WSL guidance rather than installing a Linux display driver inside WSL. Verify GPU access before loading weights.

Use either the Windows or WSL model server during the test, not both on the same port. WSL does not add VRAM. Reuse model downloads where practical, but do not reuse Windows Python environments inside Linux; Ollama stores/imports may create another copy. Native Windows remains the default path above.

</details>

</details>

Shared Python example — use after either installation:

import torch
from transformers import AutoProcessor, Qwen3_5MoeForConditionalGeneration

repo = "oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-FT"
processor = AutoProcessor.from_pretrained(repo)
model = Qwen3_5MoeForConditionalGeneration.from_pretrained(
    repo, dtype=torch.bfloat16, device_map="auto"
)
messages = [{"role": "user", "content": [
    {"type": "text", "text": "Explain gravity briefly."}
]}]
inputs = processor.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True,
    return_dict=True, return_tensors="pt", enable_thinking=False
).to(model.device)
with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=True,
        temperature=0.7,
        top_p=0.8,
        top_k=20,
        min_p=0.0,
        repetition_penalty=1.08,
    )
print(processor.batch_decode(
    output[:, inputs.input_ids.shape[1]:], skip_special_tokens=True
)[0])

For images, include an image content item supported by AutoProcessor. Large images and long contexts increase memory use.

</details>

<details> <summary>🎮 RTX 5090 · 32 GB — Q4KM with llama.cpp</summary>

Use the GGUF / Llama edition. Q4KM is about 20.22 GiB, plus the 0.84-GiB projector and runtime memory. RTX 5090 / Q4KM was benchmarked with llama.cpp commit 427291b5b34cd914a31b3fd3b61a68f6184f4b9f on Ubuntu. The Windows steps below are installation guidance, not a separate Windows 5090 result.

<details> <summary>🐧 Ubuntu / Linux — installation & launch</summary>

Install Python, the HF CLI and a CUDA-enabled llama.cpp build. Put llama-server on PATH. Choose a driver/toolkit/build supporting the RTX 5090; a CPU-only binary is not sufficient for GPU offload.

mkdir -p qwen-v2-q4
cd qwen-v2-q4
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade huggingface_hub
llama-server --list-devices
repo="oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-Llama"
hf download "$repo" Qwen3.6-35B-v2-Q4_K_M.gguf mmproj-Qwen3.6-35B-v2-F16.gguf SHA256SUMS LICENSE --local-dir .
sha256sum --check --ignore-missing SHA256SUMS
llama-server -m Qwen3.6-35B-v2-Q4_K_M.gguf   --mmproj mmproj-Qwen3.6-35B-v2-F16.gguf   -ngl 99 -c 4096 -np 1 --host 127.0.0.1 --port 8080   --reasoning off --temp 0.7 --top-p 0.8 --top-k 20 --min-p 0   --repeat-penalty 1.08 --repeat-last-n 128

Stop if checksums fail or the device listing does not show the intended NVIDIA GPU. In another terminal:

curl http://127.0.0.1:8080/v1/chat/completions   -H "Content-Type: application/json"   -d '{"messages":[{"role":"user","content":"Explain gravity briefly."}],"max_tokens":256,"temperature":0.7,"top_p":0.8,"top_k":20,"min_p":0,"repeat_penalty":1.08,"stream":false}'

</details>

<details> <summary>🪟 Windows / PowerShell — native installation & launch</summary>

Install Python 3.11 and download the matching Windows CUDA package and any required CUDA runtime DLL package from the official llama.cpp releases. Extract the full package, preserving its DLLs; add that folder to PATH for this terminal. Do not substitute a CPU-only build. These commands assume llama-server.exe is on PATH. Use PowerShell 7 for native JSON argument handling.

New-Item -ItemType Directory -Path .\qwen-v2-q4 -Force | Out-Null
Set-Location .\qwen-v2-q4
py -3.11 -m venv .venv
.\.venv\Scripts\python.exe -m pip install --upgrade huggingface_hub
llama-server.exe --list-devices
$qwenRepo = "oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-Llama"
.\.venv\Scripts\hf.exe download $qwenRepo Qwen3.6-35B-v2-Q4_K_M.gguf mmproj-Qwen3.6-35B-v2-F16.gguf SHA256SUMS LICENSE --local-dir .
if ($LASTEXITCODE -ne 0) { throw "Download failed" }
$qwenFiles = @("Qwen3.6-35B-v2-Q4_K_M.gguf", "mmproj-Qwen3.6-35B-v2-F16.gguf")
$qwenChecks = @{}
Get-Content .\SHA256SUMS | ForEach-Object {
    if ($_ -match '^([0-9a-fA-F]{64})  (.+)$') { $qwenChecks[$matches[2]] = $matches[1] }
}
foreach ($qwenFile in $qwenFiles) {
    if (-not $qwenChecks.ContainsKey($qwenFile)) { throw "Missing checksum: $qwenFile" }
    if ((Get-FileHash -LiteralPath $qwenFile -Algorithm SHA256).Hash -ne $qwenChecks[$qwenFile]) { throw "Checksum mismatch: $qwenFile" }
}
llama-server.exe -m Qwen3.6-35B-v2-Q4_K_M.gguf --mmproj mmproj-Qwen3.6-35B-v2-F16.gguf -ngl 99 -c 4096 -np 1 --host 127.0.0.1 --port 8080 --reasoning off --temp 0.7 --top-p 0.8 --top-k 20 --min-p 0 --repeat-penalty 1.08 --repeat-last-n 128

Confirm that --list-devices lists the RTX 5090. In another PowerShell window:

$qwenBody = @{
    messages = @(@{ role = "user"; content = "Explain gravity briefly." })
    max_tokens = 256
    temperature = 0.7
    top_p = 0.8
    top_k = 20
    min_p = 0.0
    repeat_penalty = 1.08
    stream = $false
} | ConvertTo-Json -Depth 5
$qwenReply = Invoke-RestMethod -Uri "http://127.0.0.1:8080/v1/chat/completions" -Method Post -ContentType "application/json" -Body $qwenBody -TimeoutSec 300
$qwenReply.choices[0].message.content

<details> <summary>🐧 Optional: Ubuntu under WSL2</summary>

Optional alternative, not required for native Windows. Follow Microsoft's WSL installation guide. From an administrator PowerShell, only if you want this alternative:

wsl --install -d Ubuntu

Restart if requested and complete Ubuntu's first-run account setup. Then open Ubuntu:

wsl -d Ubuntu

Inside Ubuntu, follow this device's Ubuntu/Linux instructions and create a separate Linux Python environment. Keep the Windows NVIDIA driver current; use NVIDIA's CUDA-on-WSL guidance rather than installing a Linux display driver inside WSL. Verify GPU access before loading weights.

Use either the Windows or WSL model server during the test, not both on the same port. WSL does not add VRAM. Reuse model downloads where practical, but do not reuse Windows Python environments inside Linux; Ollama stores/imports may create another copy. Native Windows remains the default path above.

</details>

</details>

Keep the service on localhost. For a remote server, use SSH port forwarding rather than opening an unauthenticated public endpoint. Start with a short context; other GPU workloads reduce available VRAM.

</details>

<details> <summary>💻 Laptop · 32 GB RAM / 8 GB VRAM — IQ4_XS with Ollama</summary>

IQ4_XS: about 17.86 GiB, plus a 0.84-GiB vision projector and runtime memory. It does not fit fully into 8 GB VRAM; GPU + CPU/RAM offload is required. Allow about 40 GB free disk for the download and a possible second imported copy. These are planning estimates, not measured peaks. Close memory-heavy apps and use one model at a time.

<details> <summary>🐧 Ubuntu / Linux — installation & launch</summary>

Install Ollama using its official Linux instructions, plus Python 3 and virtual-environment support. Start the Ollama service, or run ollama serve in a separate terminal if it is not already running. Do not start a second server on an occupied port.

mkdir -p qwen-v2-iq4
cd qwen-v2-iq4
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade huggingface_hub requests
repo="oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-Ollama"
hf download "$repo" Qwen3.6-35B-v2-IQ4_XS.gguf mmproj-Qwen3.6-35B-v2-F16.gguf import_ollama.py SHA256SUMS LICENSE --local-dir .
python import_ollama.py --quant IQ4_XS
ollama run qwen3.6-35b-v2:iq4_xs --think=false

</details>

<details> <summary>🪟 Windows / PowerShell — native installation & launch</summary>

Install Ollama for Windows and Python 3.11. Keep the Ollama app running. The commands use a dedicated Python environment and do not change PowerShell execution policy.

New-Item -ItemType Directory -Path .\qwen-v2-iq4 -Force | Out-Null
Set-Location .\qwen-v2-iq4
py -3.11 -m venv .venv
.\.venv\Scripts\python.exe -m pip install --upgrade huggingface_hub requests
$qwenRepo = "oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-Ollama"
.\.venv\Scripts\hf.exe download $qwenRepo Qwen3.6-35B-v2-IQ4_XS.gguf mmproj-Qwen3.6-35B-v2-F16.gguf import_ollama.py SHA256SUMS LICENSE --local-dir .
if ($LASTEXITCODE -ne 0) { throw "Download failed" }
.\.venv\Scripts\python.exe .\import_ollama.py --quant IQ4_XS
if ($LASTEXITCODE -ne 0) { throw "Import failed" }
ollama run qwen3.6-35b-v2:iq4_xs --think=false

<details> <summary>🐧 Optional: Ubuntu under WSL2</summary>

Optional alternative, not required for native Windows. Follow Microsoft's WSL installation guide. From an administrator PowerShell, only if you want this alternative:

wsl --install -d Ubuntu

Restart if requested and complete Ubuntu's first-run account setup. Then open Ubuntu:

wsl -d Ubuntu

Inside Ubuntu, follow this device's Ubuntu/Linux instructions and create a separate Linux Python environment. Keep the Windows NVIDIA driver current; use NVIDIA's CUDA-on-WSL guidance rather than installing a Linux display driver inside WSL. Verify GPU access before loading weights.

Use either the Windows or WSL model server during the test, not both on the same port. WSL does not add VRAM. Reuse model downloads where practical, but do not reuse Windows Python environments inside Linux; Ollama stores/imports may create another copy. Native Windows remains the default path above.

</details>

</details>

<details> <summary>📝 Notes — context, output & verification</summary>

The direct importer profile has a lightweight 4,096-token context and 1,024-token output cap. It is an interactive fallback, not the reliable always-on profile.

The one-command reliability installer uses 131,072 context tokens, a finite 4,096-token output cap, one request slot, non-thinking sampling and repetition protection. 9,600 tokens was used only for the reproducible Q4KM reliability regressions; it is benchmark-only and is not the normal runtime default.

The importer verifies the selected weight and projector hashes and includes both. Check CPU/GPU allocation with ollama ps. No Transformers/BF16 download is needed.

</details>

</details>

<details> <summary>🧪 Smaller experimental option — IQ2_M with Ollama</summary>

IQ2_M: about 11.70 GiB, plus a 0.84-GiB vision projector and runtime memory. It does not fit fully into 8 GB VRAM; GPU + CPU/RAM offload is required. Allow about 27 GB free disk for the download and a possible second imported copy. These are planning estimates, not measured peaks. Close memory-heavy apps and use one model at a time.

Experimental low-bit option. This is IQ2M, not Q2K. Reasoning, code and format accuracy can suffer. Initial small Windows smoke checks have started; a full laptop quality/performance comparison is still pending.

<details> <summary>🐧 Ubuntu / Linux — installation & launch</summary>

Install Ollama using its official Linux instructions, plus Python 3 and virtual-environment support. Start the Ollama service, or run ollama serve in a separate terminal if it is not already running. Do not start a second server on an occupied port.

mkdir -p qwen-v2-iq2
cd qwen-v2-iq2
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade huggingface_hub requests
repo="oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-Ollama"
hf download "$repo" Qwen3.6-35B-v2-IQ2_M.gguf mmproj-Qwen3.6-35B-v2-F16.gguf import_ollama.py SHA256SUMS LICENSE --local-dir .
python import_ollama.py --quant IQ2_M
ollama run qwen3.6-35b-v2:iq2_m --think=false

</details>

<details> <summary>🪟 Windows / PowerShell — native installation & launch</summary>

Install Ollama for Windows and Python 3.11. Keep the Ollama app running. The commands use a dedicated Python environment and do not change PowerShell execution policy.

New-Item -ItemType Directory -Path .\qwen-v2-iq2 -Force | Out-Null
Set-Location .\qwen-v2-iq2
py -3.11 -m venv .venv
.\.venv\Scripts\python.exe -m pip install --upgrade huggingface_hub requests
$qwenRepo = "oktayd/Qwen3.6-35B-v2-MoE-Ablit-Heretic-Uncensor-Hermes-MTP-Vision-Ollama"
.\.venv\Scripts\hf.exe download $qwenRepo Qwen3.6-35B-v2-IQ2_M.gguf mmproj-Qwen3.6-35B-v2-F16.gguf import_ollama.py SHA256SUMS LICENSE --local-dir .
if ($LASTEXITCODE -ne 0) { throw "Download failed" }
.\.venv\Scripts\python.exe .\import_ollama.py --quant IQ2_M
if ($LASTEXITCODE -ne 0) { throw "Import failed" }
ollama run qwen3.6-35b-v2:iq2_m --think=false

<details> <summary>🐧 Optional: Ubuntu under WSL2</summary>

Optional alternative, not required for native Windows. Follow Microsoft's WSL installation guide. From an administrator PowerShell, only if you want this alternative:

wsl --install -d Ubuntu

Restart if requested and complete Ubuntu's first-run account setup. Then open Ubuntu:

wsl -d Ubuntu

Inside Ubuntu, follow this device's Ubuntu/Linux instructions and create a separate Linux Python environment. Keep the Windows NVIDIA driver current; use NVIDIA's CUDA-on-WSL guidance rather than installing a Linux display driver inside WSL. Verify GPU access before loading weights.

Use either the Windows or WSL model server during the test, not both on the same port. WSL does not add VRAM. Reuse model downloads where practical, but do not reuse Windows Python environments inside Linux; Ollama stores/imports may create another copy. Native Windows remains the default path above.

</details>

</details>

<details> <summary>📝 Notes — context, output & verification</summary>

The direct importer profile has a lightweight 4,096-token context and 1,024-token output cap. It is an interactive fallback, not the reliable always-on profile.

The one-command reliability installer uses 131,072 context tokens, a finite 4,096-token output cap, one request slot, non-thinking sampling and repetition protection. 9,600 tokens was used only for the reproducible Q4KM reliability regressions; it is benchmark-only and is not the normal runtime default.

The importer verifies the selected weight and projector hashes and includes both. Check CPU/GPU allocation with ollama ps. No Transformers/BF16 download is needed.

</details>

</details>

<a id="additional-notes"></a> <details> <summary>➕ Additional notes — release integrity, benchmarks, architecture & quantization</summary>

<a id="release-integrity"></a> <details> <summary>🧰 Release integrity, verification & practical limits</summary>

Use SHA256SUMS to verify a downloaded GGUF or projector. RELEASE-NAMING.json is the authoritative mapping from historical v1.3 names to the current v2 filenames. RELEASE-SUMMARY.json gives the short release and smoke-test scope; GGUF-VALIDATION.json preserves the per-quant result without presenting a smoke test as a quality benchmark.

The checks establish loadability, natural EOS termination and elementary text/image smoke behavior only. They do not prove factual accuracy, coding correctness, tool reliability, visual consistency or broad quantization quality. IQ1M and IQ2M remain experimental. Thinking can loop: begin with thinking off and a finite output cap; a guard stop is incomplete output, never a successful correction.

Private training inputs, raw benchmark prompts/answers, installer archives and internal debug logs are intentionally not distributed here. Use the files and model only within applicable rights and deployment requirements.

</details>

<a id="benchmarks"></a> <details> <summary>📊 Measured diagnostics — scope matters</summary>

Scope matters. Everything in this panel is either a small local diagnostic subset, a narrowly defined local reliability regression, or an unrun plan. It is not an official leaderboard and does not support a general quality ranking. Historical Q4KM measurements, the newer validated runtime regression, and the Qwen3.8 roadmap remain intentionally separate.

Laptop runtime status. The complete IQ4_XS laptop comparison is still pending. The importer was previously tested with Ollama 0.33.3; installed versions and device-specific performance are recorded separately.

📈 Historical device diagnostics — 5 local configurations

<details> <summary>Historical totals, settings & limits — 5 local configurations</summary>

These are historical local Q4_K_M / llama.cpp diagnostics, not full official benchmark scores and not Ollama claims: temperature 0, seed 42, thinking off, 16,384 context, 4,096 output cap and 45-second per-request budget. The RTX 2000 Ada used requested 24 GPU layers plus CPU offload; the actual offload-layer log was unavailable.

How to read it: strict passes use changing denominators and omit pending/manual evaluators; ungraded is neither pass nor failure; incomplete is a separate delivery flag. Do not divide pass counts by all prompts or call them overall accuracy. The Huihui RTX 2000 Ada run left 22 tasks untested at the global deadline.

The current normal-use profile is documented separately in benchmark fixes and runtime policy: its fixed 20-row IFBench regression accepted 20/20 prompt constraints and 22/22 instruction constraints after bounded validation. That repaired regression is not comparable to the historical single-pass table above. Native decode, prefill, first-output and resource metrics remain in BENCHMARK-DEVICE-SUMMARY.json.

Qwen3.8 comparison plan registers every benchmark family from the publisher card, including internal/unavailable tasks. Qwen3.8 has not been tested locally. Publisher scores use different harnesses, settings, annotations and trial counts and are shown only as references. No GPU job is launched by these support files.

</details>

<!-- q36-capability-expansion:start -->

🔎 Per-family historical breakdowns

Expand a device to inspect every test family. Passes / graded use the original strict evaluator, including format-sensitive checks. Ungraded answers are not failures or passes. Incomplete is a separate delivery flag and can overlap with ungraded. These are small local subsets, not official leaderboard scores.

<details> <summary>Q36 · H200 — 204 attempted · 186 completed · 61 / 108 historical strict · 156.4 tok/s</summary>

</details>

<details> <summary>Q36 · RTX 5090 — 204 attempted · 190 completed · 62 / 109 historical strict · 174.6 tok/s</summary>

</details>

<details> <summary>Huihui · RTX 5090 — 204 attempted · 196 completed · 87 / 114 historical strict · 203.8 tok/s</summary>

</details>

<details> <summary>Q36 · RTX 2000 Ada (CPU+GPU) — 204 attempted · 194 completed · 61 / 109 historical strict · 18.2 tok/s</summary>

</details>

<details> <summary>Huihui · RTX 2000 Ada (CPU+GPU) — 182 attempted · 169 completed · 76 / 99 historical strict · 19.5 tok/s</summary>

</details>

Machine-readable counts · Code generation is not a pass until the relevant execution tests have been graded.

🏁 Hard benchmarks on the roadmap

All 25 families below are registered from the Qwen3.8-27B card. The matching official adapters and datasets are not yet fully prepared, and no local Qwen3.8 baseline has been run. Existing short similarly named diagnostics do not substitute for those runs.

<details> <summary>Complete plan — 25 coding, reasoning, agent and visual families</summary>

</details>

Comparison graphics will follow measured results only. Local samples and publisher-reported scores will remain separate; different harnesses, budgets and trial counts are not a head-to-head win. Full protocol and references.

💻 Coding, agents and your laptop

The next local checks cover executable coding tests, bug fixes, tool calls, planning, recovery, memory and stopping at the right time. 64 case slots are specified, including 19 existing coding slots. They are compact skill diagnostics, not proof of AGI. Results and failures will both be reported; new capability claims require actual task-level evidence.

🛠️ Laptop test plan, Hermes routing and Obsidian workflows. The runtime comparison and full memory integration are in preparation, not yet validated. The four device quickstarts above remain separate from these future end-to-end checks. <!-- q36-capability-expansion:end -->

</details>

<a id="architecture"></a> <details> <summary>⚙️ Architecture & validation — technical specifications, training scope and limits</summary>

• —Qwen3_5MoeForConditionalGeneration; approximately 35B total parameters, 256 experts, 8 selected per token (the previous A3B label).
• —Native vision-language architecture; separate projector required for GGUF image input. Text and elementary red/blue-image smoke tests were recorded. Video, 3D consistency and full desktop/browser agents are not certified by those tests.
• —Training covered selected knowledge, instruction/agent, coding, personality and visual data: 34,000 record uses across 6,000 + 12,000 + 16,000. Record uses are not unique records. Training loss is not benchmark accuracy.
• —All 19 source MTP tensors were preserved. MTP/speculative acceleration is not enabled or validated by these benchmark results. Backend support must be tested separately.
• —Tool-call formatting and end-of-answer behavior are diagnostic targets, not guaranteed features. Known schema mistakes, factual errors and repetition remain.

</details>

<a id="quantization"></a> <details> <summary>📦 Choose one quantization — 7 GGUF downloads and hardware notes</summary>

Add mmproj-Qwen3.6-35B-v2-F16.gguf (about 0.84 GiB) for image input. File size is not runtime RAM/VRAM: KV cache, activations, projector, runtime and OS require extra memory.

Q4KM/Q5KM/Q80 use standard llama.cpp quantization. IQ4XS/Q3KM/IQ1M/IQ2M use a small mixed-text importance-matrix calibration, excluding benchmark prompts. All variants were created directly from BF16, not by requantizing a low-bit file. These are not Unsloth Dynamic or Bartowski-branded exports.

IQ1_M/IQ2_M are experimental. Uncalibrated tensors, including MTP, stay at Q80; some MoE experts had incomplete calibration observations. The labels are not uniform bit widths for every weight. Substantial quality loss is possible. An IQ1M smoke answer 7 + 5 = 12 was mathematically right but violated number-only formatting; the original strict result remains available.

</details>

</details>

<a id="license"></a>

📜 License, lineage & credits

Apache-2.0 license. The training-data dropdown above summarizes source categories and attribution boundaries; model licensing does not relicense upstream datasets. Thanks to the Qwen team, the inherited model and dataset authors, and the Soup, PEFT, Transformers, llama.cpp and Ollama projects.

<details> <summary>🏷️ Name guide — model, lineage & package labels</summary>

These names do not imply affiliation, universal superiority, guaranteed compliance, removal of memorization or a promise of unrestricted behavior. Full provenance and validation remain documented below.

</details>

<details> <summary>🧬 Model lineage, checkpoint & training integrity</summary>

This release builds on the previous project's model card, which records the following stages. These are inherited stages, not new operations performed while packaging this release.

Qwen/Qwen3.6-35B-A3B
  -> lordx64 reasoning-distilled derivative
  -> huihui-ai abliterated derivative
  -> custom fused-MoE-aware Heretic stage
  -> OBLITERATUS Nuclear
  -> Hermes Function Calling + Agent/coding/terminal/file/repo/multi-tool SFT
  -> PEFT / LoRA merge
  -> Q36 BF16 master checkpoint
  -> Qwen3.6-35B v2 — current release and edition packaging
     (final GGUF exports renamed for v2; no additional weight change)

The historical SFT counts are separate from the current release's 34,000 record uses. These integrity checks establish preservation and basic load/smoke behavior; they are not laptop benchmark results or proof of general quality.

</details>

<a id="collection"></a>

🔗 Related collection

The previous release remains separate: Qwen3.6 Opus4.7 Heretic Hermes Agent — Editions.