Modelpublic

kailasa-ngpt/gemma-4-31b-tamil-devanagari-ocr-fp8

sourceHugging Faceapache-2.0updated 29d agoView on Hugging Face

0likes56downloads

Model Card

Gemma 4 31B Tamil + Devanagari OCR — FP8_BLOCK

This 32 GB vLLM-oriented artifact is quantized from the workspace's merged BF16 OCR model with LLM Compressor's Gemma 4 FP8_BLOCK recipe. Language-layer linear weights use 128x128 FP8 blocks with dynamic group-128 activations. The vision tower, multimodal projector, embeddings, and LM head remain BF16.

Use BF16 when quality is the only priority. FP8 is the compact option: on the independent 80-page gate it had essentially the same median gCER (11.01% versus 11.12%) and the same 7/80 catastrophic count, but BF16 won more paired pages (42 BF16 wins, 25 FP8 wins, 13 ties) and had slightly better clipped mean gCER. The FP8 raw mean happened to be lower because its repetition failures occurred on different pages and avoided one extreme BF16 loop; that difference was not statistically resolved.

Gate	gCER mean	gCER median	gCER clipped at 100%	Catastrophic	4,096-token caps
Independent 80-page gate	21.69%	11.01%	17.70%	7/80	4/80
Untouched base, same pages	374.74%	17.56%	25.99%	11/80	6/80

On the untouched public benchmark's 350 natural-text cases, FP8 scored 3.151% gCER (Tamil 5.366%, Devanagari 0.936%) and 53.71% exact match. Its aggregate gCER difference from the base was not statistically resolved.

vLLM

From the workspace root:

bash

OCR_MAX_NUM_SEQS=8 scripts/serve_vllm.sh \
  artifacts/gemma4-31b-tamil-devanagari-ocr-fp8
.venv-serve/bin/python scripts/ocr_vllm_client.py page.png --output page.txt

The same source-data, license, Sanskrit-script coverage, scholarly-review, and repetition-loop limitations described in the BF16 model card apply here. The artifact pins the 1,120-token image budget in both model and processor configuration, so ordinary vLLM loading matches the fine-tuning input budget.