Modelpublic

marzoukbaig14/committed-gguf-0.6b

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

1likes76downloads

Model Card

Committed — Qwen3-0.6B (Q4KM GGUF) · default model

A small model fine-tuned to write Conventional Commits messages from a git diff. It runs locally on CPU via llama.cpp, so your code never leaves your machine.

This is the default, recommended model for Committed. It started as a controlled experiment — could a model a third the size of the original 1.7B do the same job? — and it essentially can: it matches the 1.7B on picking the right commit type and staying faithful to the diff, at ~⅓ the parameters, a ~397 MB download (vs ~1 GB), and faster local inference. The one honest trade is that it writes slightly vaguer messages (lower specificity), so the larger [1.7B GGUF](https://huggingface.co/marzoukbaig14/committed-gguf) remains available as the bigger sibling if you want maximum specificity.

This repo holds the merged, quantized GGUF used for serving. The training dataset, LoRA adapter, and source code are linked below.

[Live Demo](https://my-portfolio-ten-rosy-36.vercel.app/committed) · [Gradio Space](https://huggingface.co/spaces/marzoukbaig14/committed-demo) · 0.6B GGUF (this repo) · [1.7B GGUF](https://huggingface.co/marzoukbaig14/committed-gguf) · [0.6B adapter](https://huggingface.co/marzoukbaig14/committed-qwen3-0.6b-lora) · [1.7B adapter](https://huggingface.co/marzoukbaig14/committed-qwen3-1.7b-lora) · [Dataset](https://huggingface.co/datasets/marzoukbaig14/committed-train) · [GitHub](https://github.com/marzoukbaig14/Committed)

Details

—Base: Qwen/Qwen3-0.6B (Apache-2.0)
—Method: QLoRA fine-tune (PEFT LoRA + TRL SFTTrainer, vanilla transformers), merged into the base, converted to GGUF
—Quantization: Q4KM (~397 MB / 378 MiB)
—Task: single-file git diff → one Conventional Commits subject line, type(scope): description
—Decoding: GBNF grammar-constrained, so output is always a well-formed CC line
—Trained on: marzoukbaig14/committed-train (~58k filtered CommitChronicle commits, 16 languages)

Usage

The trained behavior depends on the exact prompt rendering used in training plus the GBNF grammar applied at decode time, so a bare llama-cpp-python prompt will not reproduce the evaluated output. Run it through the project's inference path instead.

Easiest — the CLI, which downloads this GGUF on first run (no token needed; it's public) and wires up the prompt + grammar for you:

bash

pip install --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu "committed @ git+https://github.com/marzoukbaig14/Committed.git"
git diff | committed

git diff | committed uses this 0.6B model by default. To use the larger 1.7B instead:

bash

git diff | committed --model 1.7b

Or call it through the repo's engine.py, the FastAPI endpoint, or the Gradio Space — all linked below. Full instructions and options are in the repo README.

Results

Evaluated against the un-tuned Qwen3-0.6B base on a 442-example test set, scored by an LLM judge (DeepSeek) on four axes; the judge was validated against 50 hand-rated examples. Headline numbers are reweighted to the true commit-type distribution.

The four-arm comparison — both base models, both fine-tunes, all judged by the same DeepSeek judge so the numbers are directly comparable:

Metric	0.6B base	0.6B fine-tune	1.7B base	1.7B fine-tune
Type accuracy	0.154	0.601	0.131	0.637
Type-correctness	0.296	0.726	0.296	0.778
Faithfulness	0.285	0.810	0.491	0.848
Completeness	0.353	0.729	0.543	0.776
Specificity	0.414	0.545	0.814	0.667
Conjunctive (all 4)	0.101	0.359	0.175	0.471
Graded mean (0–3)	0.777	2.094	1.447	2.139
`feat`-share of outputs	86.7%	9.7%	95.5%	8.4%

The finding: both base models "feat-collapse" — they label ~87–96% of all diffs feat regardless of content, scoring below a trivial always-guess-fix baseline (0.489) on type. Fine-tuning breaks the collapse on both. The 0.6B fine-tune lands within a few points of the 1.7B on type, faithfulness, completeness, and the graded mean (2.094 vs 2.139) — the gap is concentrated in specificity (0.545 vs 0.667): the 0.6B picks the right type and states accurate facts, but names the concrete mechanism less often.

Honest caveat on the judge: agreement with human raters is moderate-to-substantial on three axes (κ ≈ 0.56–0.61) but weakest on specificity (κ 0.34) — the very axis where 0.6B trails — so that gap carries the most judge uncertainty. These DeepSeek-judged numbers are not comparable to any earlier Gemini-judged figures; only the deltas within this all-DeepSeek table are valid.

Full before/after, the feat-collapse analysis, and the judge validation are in the eval writeup: [FINDINGS_v1.md](https://github.com/marzoukbaig14/Committed/blob/main/docs/eval/FINDINGS_v1.md).

—The larger 1.7B GGUF (bigger sibling, higher specificity) — marzoukbaig14/committed-gguf.
—This model's LoRA adapter — marzoukbaig14/committed-qwen3-0.6b-lora. The unmerged adapter weights, if you want to merge or train further.
—Live demo (browser) — try it on the portfolio site. Paste a diff, pick a model, get a commit message.
—Live demo (Gradio) — Hugging Face Space.
—Training dataset — marzoukbaig14/committed-train, filtered from CommitChronicle, with composition tables and provenance.
—Source, training & eval code — github.com/marzoukbaig14/Committed.
—Eval writeup — FINDINGS_v1.md.

License

Apache-2.0, inherited from the Qwen3-0.6B base.

Citation