Modelpublic

gnitoahc/ceed-b3

sourceHugging Facegemmaupdated 1mo agoView on Hugging Face

0likes4downloads

Model Card

CEED B3 — gemma-4-e4b-it, distilled, with hidden-state projection

A LoRA fine-tune of `google/gemma-4-e4b-it` trained with B2's objective plus hidden-state projection distillation at three mapped layers. The teacher is the sparse mixture-of-experts `google/gemma-4-26b-a4b-it`.

The adapter has been folded into the base weights, so this is a standalone checkpoint: load it exactly like the base model, with no PEFT and no CEED code.

This is Group B3 of the CEED study (Causal Expert–Evidence Distillation), a research artifact published for reproducibility. It is not a product.

Usage

python

from transformers import AutoModelForImageTextToText, AutoProcessor

model = AutoModelForImageTextToText.from_pretrained("gnitoahc/ceed-b3", dtype="float16")
processor = AutoProcessor.from_pretrained("gnitoahc/ceed-b3")

The model was trained and scored with a short-answer instruction in the prompt. Without it an instruction-tuned model answers "The total written in the image is **28**." against gold "28" and scores zero on every metric here.

Training


Corpus	chartqa 2,500, docvqa 5,349, gqa 10,000 (17,849 examples, 80/10/10 split by example id)
Passes over the training split	2.69
Adapter	LoRA rank 4
Final cross-entropy	0.9126
Final KD term	2.4891
Seed	0
Run identity	`51223b3cbed5ef07221c2dabd183a1a706ed42e0fd24595c2d87ffd9b80342fe`

Evaluation

Dataset	Metric	Score	n
docvqa	ANLS	0.8501	565
gqa	exact match	0.6289	1016
chartqa	relaxed accuracy	0.5341	249

Scored by CEED's own harness (harness_version: ceed-direct-1) with greedy decoding, on CEED's own 10% validation split.

These numbers are not comparable to published DocVQA / GQA / ChartQA leaderboard results. Different splits, different prompt, different decoding. They are meaningful only against the other CEED Groups, which were scored identically.

Limitations

—This is a LoRA result. Merging folds the adapter into the weights; it does not turn a rank-4 adapter into a full fine-tune. CEED's own ADR-0005 bars LoRA numbers from the study's headline table, because a null result under a small adapter cannot be attributed between "the signal does not transfer" and "the adapter lacked the capacity to hold it". Read any comparison involving this checkpoint with that in mind.
—The distillation gain is not established. The no-teacher control (CEED B1), trained identically but with kd_weight: 0, scored above this checkpoint on every dataset (docvqa 0.8501 vs 0.8798; gqa 0.6289 vs 0.6959; chartqa 0.5341 vs 0.7871). Whatever this checkpoint's objective contributes, it is not visible as an advantage over supervised fine-tuning here.
—Trained on document, natural-image and chart VQA in English only. Behaviour outside that is untested.
—Inherits the base model's limitations and the Gemma licence.

ceed_provenance.json beside the weights carries the source run's identity, parameter-efficiency mode, and metrics.