hugging-apps/bar-jepa-chart-extraction

Bar-JEPA — Extracting Values from Bar Charts

Interactive demo of Bar-JEPA: Extracting Values from Bar Chart with Joint-Embedding Predictive Architecture (Poonam, Epple & Ropinski, Ulm University — paper, code).

The pipeline recovers the per-bar numerical values of a vertical bar chart:

Image → variable-resolution patches (≤256, 14 px, aspect-ratio preserving)
      → I-JEPA ViT-H encoder
      → classic keypoint decoder (cls / reg / origin heads)
      → NMS → PaddleOCR tick labels → RANSAC → bar values

Upload a vertical bar chart, press Extract values, and inspect:

• —Detections — the chart with predicted bar keypoints (× green) and value-axis ticks (× orange) overlaid.
• —Activation maps — sigmoid classification maps for background / bar / tick.
• —Recovered values — a table of bars sorted left → right, with the value read off the OCR + RANSAC regression of the tick labels.

This Space runs the kp-cl-arp-ctt-ft-latest checkpoint (classic decoder, ARP encoder with Chart-to-Text finetuning) — the best configuration reported in the paper.

Notes & limitations

• —Vertical bar charts only — no stacked bars, error bars or 3D effects (the model was trained on synthetic charts of this style).
• —The oracle path of the paper's evaluation (using ground-truth tick labels) is not available for user uploads, so values come from the OCR path only.
• —Example charts are from the `dralois/Bar-JEPA` pipeline_testing split.

License & attribution

Model weights, dataset and example images: `dralois/Bar-JEPA` (CC-BY-NC-4.0). Model code derived from facebookresearch/ijepa.