C3DS/CARDS-Qwen3.5-4B-GGUF

CARDS-Qwen3.5-4B-GGUF

GGUF builds of `C3DS/CARDS-Qwen3.5-4B` — Qwen3.5-4B fine-tuned for climate-contrarian-claim classification under the CARDS taxonomy from Coan et al. (2025).

For accuracy comparisons against the BF16 sibling, larger Qwen3 variants, Claude Opus 4.6 / 4.7 and GPT-5.5, see the BF16 model card. Headline number on the held-out CARDS test set: L1 samples F1 = 0.838 (1 / 1,436 parse failures).

Available files

The Q4_K_M and Q8_0 files are alternatives — pick one based on your accuracy / memory trade-off. The mmproj file is additional and only needed for image inputs.

Usage

With Ollama

A pre-published Ollama version is available at `exec3ds/cards-qwen3.5`. Pull and run the 4B (Q4KM, ~2.7 GB):

ollama run exec3ds/cards-qwen3.5:4b

The same namespace also publishes :9b and :27b tags for the larger siblings. The bundled chat template is applied automatically — Qwen3's <think> block precedes the YAML categories: output.

Image input works too — pass an image path on the command line:

ollama run exec3ds/cards-qwen3.5:4b ./image.webp \
  "What climate claim is being made in this image?"

Text-only with llama.cpp

llama-cli \
  -hf C3DS/CARDS-Qwen3.5-4B-GGUF:Q4_K_M \
  --jinja \
  -p "Classify the following climate claim under the CARDS taxonomy:\nThese are only a few renewable energy technologies at work."

The --jinja flag tells llama-cli to use the bundled chat template, which adds Qwen3's <think> reasoning trace plus the YAML categories: block at the end. Parse the content after </think> to extract the predicted CARDS codes.

For an OpenAI-compatible local server, use llama-server:

llama-server \
  -hf C3DS/CARDS-Qwen3.5-4B-GGUF:Q4_K_M \
  --jinja \
  --port 8000

Multimodal — image + text

The base Qwen3.5 architecture (Qwen3_5ForConditionalGeneration) is multimodal. Pair the weight file with the BF16-mmproj.gguf projector and use llama-mtmd-cli:

llama-mtmd-cli \
  -hf C3DS/CARDS-Qwen3.5-4B-GGUF:Q4_K_M \
  --mmproj-hf C3DS/CARDS-Qwen3.5-4B-GGUF/Qwen3.5-4B.BF16-mmproj.gguf \
  --jinja \
  --image screenshot.png \
  -p "Read the image (and any caption) and classify the climate claim under the CARDS taxonomy."

For the canonical CARDS system prompt and CoT trigger, fetch `cards_prompts.json` from the BF16 sibling repo and pass slim_system_instruction as the system prompt + cot_trigger at the end of the user turn.

Conversion details

• —Source checkpoint: `C3DS/CARDS-Qwen3.5-4B` (LoRA-merged Qwen3.5-4B)
• —Tool: Unsloth → llama.cpp converter
• —Quantizations included: Q4KM, Q8_0
• —Multimodal projector: BF16 (kept full-precision for accuracy on vision inputs)

<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>

Limitations

• —Thinking tokens. Training used enable_thinking=True. Either parse output after </think>, or disable thinking in your sampler / system prompt. Reserve token budget for the reasoning trace before the final YAML block.
• —Quantization trade-offs. Q4KM is the most aggressive; expect a modest drop from BF16 in the long-tailed L3 macro-F1 metrics. Q8_0 tracks BF16 more closely.
• —Multimodal use is opportunistic. Training was text-only; image-input behavior comes from the preserved base vision encoder. Performance on image inputs has not been quantitatively benchmarked.

Citation

@article{coan2025cards,
  title   = {Large language model reveals an increase in climate contrarian speech in the United States Congress},
  author  = {Coan, Travis G. and Malla, Ranadheer and Nanko, Mirjam O. and Kattrup, William and Roberts, J. Timmons and Cook, John and Boussalis, Constantine},
  journal = {Communications Sustainability},
  volume  = {1},
  pages   = {37},
  year    = {2025},
  doi     = {10.1038/s44458-025-00029-z}
}

License

Apache 2.0, inherited from Qwen3.5-4B.