TomasFAV/Pix2StructCzechInvoiceV0

Pix2StructCzechInvoice (V0 – Synthetic Templates Only)

This model is a fine-tuned version of google/pix2struct-docvqa-base for structured information extraction from Czech invoices.

It achieves the following results on the evaluation set:

• —Loss: 0.5022
• —F1: 0.5907

Model description

Pix2StructCzechInvoice (V0) is a generative multimodal model designed for document understanding.

Unlike token classification models (e.g., BERT, LiLT, LayoutLMv3), this model:

• —processes the entire document image
• —generates structured outputs as text sequences

The model is trained to extract key invoice fields such as:

• —supplier
• —customer
• —invoice number
• —bank details
• —totals
• —dates

Training data

The dataset consists of:

• —synthetically generated invoice images
• —fixed template layouts
• —corresponding target text sequences representing structured fields

Key properties:

• —clean and consistent visual structure
• —no OCR noise (end-to-end image input)
• —controlled output formatting
• —no real-world documents

This represents the baseline dataset for generative multimodal models.

Role in the pipeline

This model corresponds to:

V0 – Synthetic template-based dataset only

It is used to:

• —establish a baseline for generative document models
• —compare with:
• —token classification approaches (BERT, LiLT)
• —multimodal encoders (LayoutLMv3)
• —evaluate feasibility of end-to-end extraction

Intended uses

• —End-to-end invoice information extraction from images
• —Document VQA-style tasks
• —Research in generative document understanding
• —Comparison with structured prediction approaches

Limitations

• —Trained only on synthetic data
• —Sensitive to output formatting inconsistencies
• —Lower stability compared to token classification models
• —Requires careful evaluation (string matching vs structured metrics)
• —Performance depends on generation quality

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• —learning_rate: 0.0001
• —trainbatchsize: 4
• —evalbatchsize: 1
• —seed: 42
• —optimizer: Use OptimizerNames.ADAMWTORCHFUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• —lrschedulertype: cosinewithrestarts
• —lrschedulerwarmup_steps: 0.1
• —num_epochs: 10
• —mixedprecisiontraining: Native AMP

Training results

Framework versions

• —Transformers 5.0.0
• —PyTorch 2.10.0+cu128
• —Datasets 4.0.0
• —Tokenizers 0.22.2