Datasetpublic

vi-ocean/SymVAE

Geometric Logic-Form Dataset (SymParser / SymVAE / SymHPR) Dataset for the CVPR 2026 paper Hierarchical Process Reward Models are Symbolic Vision Learners — Shan Zhang, Aotian Chen, Kai Zou, Jindong Gu, Yuan Xue, Anton van den Hengel. 🔗 Project page: vi-ocean.github.io/projects/SymVAE — SymVAE stands for Symbolic Variational Auto-Encoder. A multimodal dataset for training and evaluating MLLMs on geometric diagram → logic-form extraction. Given a geometry image, a model must… See the full description on the dataset page: https://huggingface.co/datasets/vi-ocean/SymVAE.

sourceHugging Facecc-by-4.0updated 15d agoView on Hugging Face

0likes354downloads

Dataset Card

Geometric Logic-Form Dataset (SymParser / SymVAE / SymHPR)

Dataset for the CVPR 2026 paper **Hierarchical Process Reward Models are Symbolic Vision Learners** — Shan Zhang, Aotian Chen, Kai Zou, Jindong Gu, Yuan Xue, Anton van den Hengel.

🔗 Project page: vi-ocean.github.io/projects/SymVAE — SymVAE stands for Symbolic Variational Auto-Encoder.

A multimodal dataset for training and evaluating MLLMs on geometric diagram → logic-form extraction. Given a geometry image, a model must produce a structured logic form describing the diagram: point instances, line instances, shape instances, geometric relations, and normalized point positions.

The release contains three subsets:

Subset	Purpose	Train	Eval	Format
SymParser_100K	Supervised fine-tuning (SFT)	106,276	—	LLaVA conversation
SymVAE_16K	RL / reward modeling	16,000	2,063	OpenAI messages
SymHPR_9K	RL / preference (with logic-form GT)	9,000	1,000	OpenAI messages

All images live under images/ and are shared across the three subsets (3.4 GB total, 135,172 PNGs).

Directory Structure

Because the image collection contains ~141k files (more than a git host allows per directory), the images are shipped as two tar archives. Everything else is plain JSON.

.
├── images_SFT.tar                   # 2.6 GB → extracts to images/SFT/images/   (106,276 png, SymParser)
├── images_RL.tar                    # 619 MB → extracts to images/RL/...         (RL images)
│                                    #   RL/images (11,865, LogicForm) + RL/annotated_images
│                                    #   (5,000, PGDP) + RL/geo170k (18,040, Geo170K: geo3k / geoqa_plus)
├── SymParser_100K/
│   └── SymParser_100K.json
├── SymVAE_16K/
│   ├── SymVAE_train.json
│   └── SymVAE_eval.json
└── SymHPR_9K/
    ├── SymHPR_train_with_logicform.json
    └── SymHPR_eval_with_logicform.json

Setup — extract the images once

bash

# after downloading the repo (see Usage below)
tar xf images_SFT.tar     # → images/SFT/images/...
tar xf images_RL.tar      # → images/RL/...

This recreates a single images/ directory shared by all three subsets (3.4 GB total, 135,172 PNGs).

⚠️ Image path convention. The image / image_path fields in the JSON files are written relative to the images/ directory, e.g. SFT/images/final_construction_sample_0.png or RL/images/final_construction_sample_11021.png. After extracting the tars, resolve a full path by prepending images/: full_path = os.path.join("images", record_image_path)

Subsets

1. SymParser_100K (SFT)

LLaVA-style conversations. Field image points to SFT/images/*.png.

json

{
  "id": "logic_form_1",
  "image": "SFT/images/final_construction_sample_0.png",
  "conversations": [
    {"from": "human", "value": "<image>\nGenerate the logic forms for this image."},
    {"from": "gpt", "value": "Point instances: [l, g, h, w, s, x, n, v, b]\n\nLine instances: [sx, xn, nv, vs]\n\nShape instances:\nCircle(b)\nSquare(s, x, n, v)\n..."}
  ]
}

2. SymVAE_16K (RL)

OpenAI-messages-style prompts. Sources: Geo170K (7,000), LogicForm (5,000), PGDP (4,000).

json

{
  "prompt": [
    {"role": "user", "content": [
      {"type": "image_url", "image_url": {"url": "RL/images/final_construction_sample_11021.png"}},
      {"type": "text", "text": "Analyze this geometric diagram and extract a complete logic form."}
    ]}
  ],
  "answer": "Point instances: [q, k, p, o]\n\nLine instances: [qk, kp, pq]\n...",
  "info": {
    "image_path": "RL/images/final_construction_sample_11021.png",
    "source": "LogicForm",
    "idx": 5000,
    "original_id": "stage1_logic_form_10477",
    "image_filename": "RL/images/final_construction_sample_11021.png",
    "logic_form_gt": { "point_instances": [...], "line_instances": [...], "shape_instances": [...], "indicator_of_shape": null, "geometric_relations": [...], "point_positions": {...} }
  }
}

3. SymHPR_9K (RL, with logic-form GT)

Same schema as SymVAE. Sources: LogicForm (5,000), PGDP (4,000). Every record includes structured logic_form_gt in info.

Logic-Form Schema

Answers are text but follow a fixed structure, which info.logic_form_gt mirrors as fields:

—Point instances — [q, k, p, o]
—Line instances — [qk, kp, pq]
—Shape instances — Circle(o), Triangle(q, k, p), Square(...)
—Indicator of shape — constraints (Parallel, Perpendicular, Equals(LengthOf(...), ...))
—Geometric relations — PointLiesOnCircle(q, Circle(o, radius_0_0)), ConcyclicPoints(...)
—Point positions — normalized coordinates in [0, 1], e.g. q=(0.225, 0.547)

Usage

Download the whole repo, then extract the two image tars once:

python

from huggingface_hub import snapshot_download
snapshot_download(repo_id="vi-ocean/SymVAE", repo_type="dataset", local_dir="./data")

bash

cd ./data
tar xf images_SFT.tar     # → images/SFT/images/...
tar xf images_RL.tar      # → images/RL/...

Then load a subset (image paths are relative to the extracted images/ folder):

python

import json, os

root = "./data"  # local clone / snapshot dir (after extracting the tars)

# --- SFT ---
sft = json.load(open(os.path.join(root, "SymParser_100K/SymParser_100K.json")))
rec = sft[0]
img = os.path.join(root, "images", rec["image"])   # prepend images/

# --- RL ---
rl = json.load(open(os.path.join(root, "SymVAE_16K/SymVAE_train.json")))
rec = rl[0]
img = os.path.join(root, "images", rec["info"]["image_path"])

Data Sources

Images are aggregated / rendered from: synthetically constructed geometry diagrams (LogicForm), PGDP (annotated_images), and Geo170K (geo3k, geoqa_plus). Please cite the original datasets when using the corresponding subsets.

License

Released under CC-BY-4.0 for the newly created annotations. Third-party image sources (Geo170K, PGDP) retain their original licenses.

Citation

bibtex

@inproceedings{zhang2026hierarchical,
  title     = {Hierarchical Process Reward Models are Symbolic Vision Learners},
  author    = {Zhang, Shan and Chen, Aotian and Zou, Kai and Gu, Jindong and Xue, Yuan and van den Hengel, Anton},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  pages     = {22185--22194},
  year      = {2026}
}