JoaoZaokk/Qwen3-4B-Thinking-2507-Heretic-CodeFeedback-OpenVINO-INT4

Qwen3 4B Thinking 2507 Heretic CodeFeedback — OpenVINO INT4

This repository contains the OpenVINO INT4 export of:

`JoaoZaokk/Qwen3-4B-Thinking-2507-Heretic-CodeFeedback`

The source model is a merged code-focused fine-tune based on:

`JoaoZaokk/Qwen3-4B-Thinking-2507-MiniMax-M2.1-Distill-heretic`

This version was converted from the full merged safetensors model to OpenVINO IR with INT4 weight compression, intended primarily for Intel Arc / OpenVINO / OpenVINO GenAI inference.

This is my first merged model, thats basically for testing, i'll update it when i have time to.

Related repositories

Format

Conversion command

optimum-cli export openvino \
  --model ~/models-src/Qwen3-4B-Thinking-2507-Heretic-CodeFeedback \
  --task text-generation-with-past \
  --weight-format int4 \
  --group-size 128 \
  --ratio 1.0 \
  --trust-remote-code \
  ./Qwen3-4B-Thinking-2507-Heretic-CodeFeedback-OpenVINO-INT4

During conversion, most weights were compressed to INT4:

int4_asym, group size 128

A small portion may remain in INT8 depending on OpenVINO/NNCF layer handling.

Tested local inference

This export was tested with OpenArc / OpenVINO GenAI on:

Example observed metrics from a short Python-code prompt:

OpenArc load example

curl -X POST http://localhost:8000/openarc/load \
  -H "Content-Type: application/json" \
  -d '{
    "model_path": "/models/Qwen3-4B-Thinking-2507-Heretic-CodeFeedback-OpenVINO-INT4",
    "model_name": "Qwen3-4B-Thinking-2507-Heretic-CodeFeedback-OpenVINO-INT4",
    "model_type": "llm",
    "engine": "ovgenai",
    "device": "GPU.0",
    "runtime_config": {}
  }'

Training background

The source model was trained with QLoRA/LoRA on Python and code instruction datasets, then merged back into the base model.

A SWE-smith trajectory experiment was tested separately, but it was not used in the final merged version.

LoRA configuration of source model

Target modules:

• —q_proj
• —k_proj
• —v_proj
• —o_proj
• —gate_proj
• —up_proj
• —down_proj

Intended use

This INT4 OpenVINO version is intended for:

• —local Intel Arc inference
• —OpenVINO GenAI experiments
• —Python code generation
• —code explanation
• —simple debugging
• —instruction-following tests
• —low-VRAM local inference compared to the F16 merged model

Hardware notes

Important notes

This is not the original F16 model. This is an OpenVINO INT4 compressed export.

For further conversions, use the original merged safetensors repository as the master source:

`JoaoZaokk/Qwen3-4B-Thinking-2507-Heretic-CodeFeedback`

This is an experimental model. It may produce incorrect code, unsafe suggestions, or hallucinated explanations. Outputs should be reviewed before use in production or security-sensitive environments.