Modelpublic

marinarosa/minicpmv4.6-vivamais-v1

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes

Model Card

MiniCPM-V 4.6 — Viva Mais PT-BR Document Extractor (v1)

A MiniCPM-V 4.6 (1.3B) vision-language model LoRA-fine-tuned for accent-faithful Brazilian-Portuguese travel-document understanding: it transcribes a document photo, classifies it into one of ten kinds, and extracts typed fields with Brazilian formats (R$ 1.234,56, DD/MM/AAAA, CPF/CNPJ).

Two backends are published:

—this repo — merged Transformers weights under sft_v4/;
—marinarosa/minicpmv4.6-vivamais-v1-GGUF — Q4KM + mmproj for llama.cpp.

Training data — 100% synthetic, privacy-safe

v1 is trained only on synthetic documents (no real client data). Person names are drawn from accent-dense Brazilian pools screened against a real-name blocklist; CPF/CNPJ, values, codes, routes, and dates are generated. The renderer guarantees diacritic-capable TrueType fonts and applies phone-photo augmentation (perspective, glare, JPEG recompression). Dataset: `marinarosa/vivamais-synthetic-ptbr`.

Evaluation — held-out real documents

Evaluated on 346 real Brazilian travel documents (a true held-out set: v1 saw no real data in training), scored against teacher (Qwen3-VL) labels. Stage-2 field scoring is split because the gold is ~49% empty: recall over populated gold fields measures extraction; over-fill over empty gold fields measures invention.

metric	base MiniCPM-V 4.6	v1
kind accuracy	0.006	0.584
Stage-2 field recall (populated)	0.024	0.177
accent-preservation rate	0.622	0.630
transcription CER (↓ better)	0.562	0.545
BR-format field accuracy	0.810	0.884
Stage-2 over-fill (↓ better)	0.007	0.594

v1 recovers ~7× more present fields than the zero-shot base and wins on classification, accents, transcription, and BR formats.

Known limitation — over-fill

v1 over-fills: it invents a value in ~59% of fields that are absent from the document, because every synthetic training example populated every field. If your pipeline trusts every returned field, post-filter low-confidence values. The next version (v2) is trained with optional-field dropout to teach restraint.

Intended use & scope

Local-first extraction of Brazilian travel documents (air quotes/reservations, boarding passes, payment proofs, invoices, accommodation). Built for the Hugging Face × Gradio "Build Small" hackathon (runs fully offline via llama.cpp). Not a general OCR model; the field schema is the Viva Mais 10-kind / 7-payload domain. Outputs may be wrong — verify before acting on extracted values.

Usage (Transformers)

python

from transformers import AutoModel, AutoProcessor
model = AutoModel.from_pretrained(
    "marinarosa/minicpmv4.6-vivamais-v1", subfolder="sft_v4",
    trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained(
    "marinarosa/minicpmv4.6-vivamais-v1", subfolder="sft_v4",
    trust_remote_code=True,
)

For llama.cpp use the GGUF repo with the Q4KM model + the mmproj file.

License

Apache-2.0, inheriting the openbmb/MiniCPM-V-4.6 base license.