Modelpublic

gnitoahc/ceed-b1

sourceHugging Facegemmaupdated 1mo agoView on Hugging Face

0likes3downloads

Model Card

CEED B1 — gemma-4-e4b-it, supervised fine-tuning, no teacher

A LoRA fine-tune of `google/gemma-4-e4b-it` trained with cross-entropy on gold answers only, with no teacher -- the control that says how much of a distilled Group's gain is distillation rather than fine-tuning.

The adapter has been folded into the base weights, so this is a standalone checkpoint: load it exactly like the base model, with no PEFT and no CEED code.

This is Group B1 of the CEED study (Causal Expert–Evidence Distillation), a research artifact published for reproducibility. It is not a product.

Usage

python

from transformers import AutoModelForImageTextToText, AutoProcessor

model = AutoModelForImageTextToText.from_pretrained("gnitoahc/ceed-b1", dtype="float16")
processor = AutoProcessor.from_pretrained("gnitoahc/ceed-b1")

The model was trained and scored with a short-answer instruction in the prompt. Without it an instruction-tuned model answers "The total written in the image is **28**." against gold "28" and scores zero on every metric here.

Training


Corpus	chartqa 2,500, docvqa 5,349, gqa 10,000 (17,849 examples, 80/10/10 split by example id)
Passes over the training split	2.69
Adapter	LoRA rank 4
Final cross-entropy	0.5048
Final KD term	0.0000
Seed	0
Run identity	`f973d6eeb743e5b5b9b4d7512da2b37665a519c980afa2fb88f3abc54c5a1706`

Evaluation

Dataset	Metric	Score	n
docvqa	ANLS	0.8798	565
gqa	exact match	0.6959	1016
chartqa	relaxed accuracy	0.7871	249

Scored by CEED's own harness (harness_version: ceed-direct-1) with greedy decoding, on CEED's own 10% validation split.

These numbers are not comparable to published DocVQA / GQA / ChartQA leaderboard results. Different splits, different prompt, different decoding. They are meaningful only against the other CEED Groups, which were scored identically.

Limitations

—This is a LoRA result. Merging folds the adapter into the weights; it does not turn a rank-4 adapter into a full fine-tune. CEED's own ADR-0005 bars LoRA numbers from the study's headline table, because a null result under a small adapter cannot be attributed between "the signal does not transfer" and "the adapter lacked the capacity to hold it". Read any comparison involving this checkpoint with that in mind.
—This Group has no teacher. B1 is the study's control: it isolates how much of a distilled Group's gain is distillation rather than plain fine-tuning. It is not itself a distillation result.
—Trained on document, natural-image and chart VQA in English only. Behaviour outside that is untested.
—Inherits the base model's limitations and the Gemma licence.

ceed_provenance.json beside the weights carries the source run's identity, parameter-efficiency mode, and metrics.