cstr/posformer-crohme-GGUF

PosFormer Handwritten Math OCR — CROHME-trained GGUF

PosFormer (Position-aware Transformer) for handwritten mathematical expression recognition, retrained from scratch on CROHME 2014 and converted to GGUF format for CrispEmbed.

License

CC BY-NC-SA 3.0 — inherited from the CROHME 2014 training data.

• —Personal, educational, and research use: allowed
• —Apps (including commercial): allowed — the app is a separate work; the NC clause applies to the weights, not to software that loads them. Users download the weights separately and accept the NC terms.
• —Redistributing or selling the weights themselves: not allowed
• —Attribution required: cite CROHME and PosFormer (see below)
• —ShareAlike: derivative models must use the same or compatible license

The C++ inference engine (CrispEmbed) and GGUF converter are original clean-room implementations (MIT license).

Model details

Training details

Retrained from scratch (no transfer learning from published weights) using the PosFormer architecture on CROHME 2014 train set. Key differences from the published training:

• —Cosine annealing with warm restarts (T0=30, Tmult=2) instead of ReduceLROnPlateau
• —Label smoothing (ε=0.1) on cross-entropy loss
• —Greedy validation (beam_size=1) for faster training epochs

Training monitored via Weights & Biases.

Files

Accuracy (CROHME 2014 test set, 986 images)

Greedy left-to-right decoding (beam_size=1, no bi-directional search):

Trained on CROHME 2014 (8835 images) + 2000 MathWriting samples (filtered to 110-token vocab). 60.5% beam=1 exact match on full CROHME 2014 test (986 images). val_ExpRate peaked at 62.0% during training. Surpasses published SJTU greedy (56.0%) by 4.5 points. LR=0.00125 after ReduceLROnPlateau drop.

Note: the published PosFormer ExpRate of 62.7% uses bi-directional beam search (beam_size=10). Our greedy results are directly comparable to our C++ inference engine which uses greedy L2R decoding.

Usage with CrispEmbed

# Build
cd CrispEmbed-build
cmake /path/to/CrispEmbed
make -j$(nproc) test-posformer

# Run
export LD_LIBRARY_PATH=$PWD/ggml/src
./test-posformer posformer-crohme-q8_0.gguf image.bmp

Parity

The C++ inference engine matches PyTorch to >99.999% (cosine similarity = 1.000000 at every decoder step). See tests/parity/posformer_*.py in the CrispEmbed repo for verification scripts.

Citation

@inproceedings{chen2024posformer,
  title={PosFormer: Recognizing Complex Handwritten Mathematical Expression
         with Position Forest Transformer},
  author={Chen, Tongkun and others},
  booktitle={AAAI},
  year={2024}
}

References

• —PosFormer paper (AAAI 2024)
• —Original code: SJTU-DeepVisionLab/PosFormer
• —CrispEmbed inference engine
• —CROHME competition

Provenance — authored here, and outside the GPAI definition

Added 2026-08-02 during an account-wide provenance review.

These weights were trained by this repository's maintainer, not converted from someone else's model. Most cstr/* repositories are GGUF/ONNX conversions where the upstream research team remains the provider; this one is not, and the distinction matters because the two attract different obligations. The training data, hyperparameters and licence inheritance are documented above.

EU AI Act Art. 53 does not apply. Art. 53 binds providers of general-purpose AI models, which Art. 3(63) defines as models displaying "significant generality" and capable of "competently performing a wide range of distinct tasks". PosFormer recognising handwritten mathematical expressions is a narrow, single-task model and does not meet that definition, so the Art. 53(1)(c) copyright-policy and 53(1)(d) training-content duties are not engaged. This is recorded explicitly because "trained here" and "subject to Art. 53" are easy to conflate, and only the first is true.

The training-data documentation above is published because it is useful and because the licence inheritance depends on it — not because Art. 53 compels it.