alwaysgood/qwen35_sft_023_dpo-GGUF

qwen35sft023_dpo GGUF

GGUF conversion of `alwaysgood/qwen35_sft_023_dpo` for llama.cpp and LM Studio.

Files

The main BF16 and Q4_K_M GGUF files are language/text models. Vision/image tensors are split into the separate mmproj-BF16 GGUF, so image input requires loading both the main model and the matching mmproj file.

LM Studio custom template

This model was trained with a raw DPO/SFT translation prompt, not a ChatML chat prompt. LM Studio's normal User/Assistant split can still be used by putting the fixed Instruction and Source headers in the custom template.

In LM Studio, open the model settings, go to Advanced Configuration -> Prompt Template, choose Custom, and use:

If you type this in LM Studio chat:

The Federal Reserve raised interest rates by 25 basis points.

LM Studio should send this prompt to the model:

### Instruction:
Translate the English source into Korean.

### Source:
The Federal Reserve raised interest rates by 25 basis points.

### Response:

Suggested decoding: greedy or low-temperature translation, e.g. temperature 0.0 to 0.3, context length 8192.

llama.cpp

Text:

llama-cli \
  -m qwen35_sft_023_dpo-Q4_K_M.gguf \
  --color -ngl all -fa -c 8192 \
  -p $'### Instruction:\nTranslate the English source into Korean.\n\n### Source:\nThe Federal Reserve raised interest rates by 25 basis points.\n\n### Response:\n'

Vision:

llama-mtmd-cli \
  -m qwen35_sft_023_dpo-Q4_K_M.gguf \
  --mmproj qwen35_sft_023_dpo-mmproj-BF16.gguf \
  --image /path/to/image.png \
  -p $'### Instruction:\nTranslate the English source into Korean.\n\n### Source:\nDescribe this image in Korean.\n\n### Response:\n' \
  -ngl all -fa -c 8192

Notes

The source checkpoint contains both language and vision tensors. The GGUF text model was converted with the vision tensors split out, and the multimodal projector was converted separately.

Smoke test result with llama.cpp:

Q: 1+1=? A: 2