HuggingAI4Engineering/cadgenbench-data

1---2license: odc-by3language:4- en5pretty_name: CADGenBench (Inputs)6task_categories:7- image-to-3d8- text-to-3d9tags:10- cad11- 3d12- step13- mechanical-engineering14- benchmark15- cad-generation16- cad-editing17size_categories:18- n<1K19annotations_creators:20- expert-generated21language_creators:22- expert-generated23multilinguality:24- monolingual25source_datasets:26- original27viewer: false28---29 30# CADGenBench (Inputs)31 32Public inputs for the **CADGenBench** benchmark, which measures how well AI33systems produce correct 3D mechanical parts as STEP files. This repository34holds the task inputs only; the ground truth is withheld in a separate private35repository so that the leaderboard's evaluation is the single source of truth.36 37- **Leaderboard Space**: [`HuggingAI4Engineering/CADGenBench`](https://huggingface.co/spaces/HuggingAI4Engineering/CADGenBench)38- **Browse the tasks**: the [Tasks tab](https://huggingface.co/spaces/HuggingAI4Engineering/CADGenBench) on the Space (thumbnails, search, generation/editing filter, per-task detail)39- **Benchmark code**: [`github.com/huggingface/cadgenbench`](https://github.com/huggingface/cadgenbench)40- **Submissions + results**: [`HuggingAI4Engineering/cadgenbench-submissions`](https://huggingface.co/datasets/HuggingAI4Engineering/cadgenbench-submissions)41- **Ground truth**: [`HuggingAI4Engineering/cadgenbench-data-gt`](https://huggingface.co/datasets/HuggingAI4Engineering/cadgenbench-data-gt) (private)42 43## Dataset Summary44 45CADGenBench contains **81 fixtures** of mechanical parts split across two46tasks:47 48- **Generation (49)**: reproduce the part as an accurate 3D solid from an49  engineering drawing.50- **Editing (32)**: apply a described change to an existing STEP solid.51 52Each submission is one `output.step` per fixture. Outputs are scored against53the private ground truth by the CAD Score pipeline: a hard validity gate54followed by a weighted mean of three orthogonal metrics (shape similarity,55interface match, topology match). See the56[metric definitions](https://github.com/huggingface/cadgenbench/blob/main/docs/metrics.md).57 58## Supported Tasks and Leaderboards59 60The benchmark is tool-agnostic: a submission is one STEP file per fixture,61produced by any system (one LLM, several, a script, or by hand). Submit and62view results on the63[leaderboard Space](https://huggingface.co/spaces/HuggingAI4Engineering/CADGenBench);64the full submission contract (zip layout, `meta.json`, validity gate) is in65[`docs/benchmark/submission.md`](https://github.com/huggingface/cadgenbench/blob/main/docs/benchmark/submission.md).66 67## Languages68 69Task prompts and descriptions are in English.70 71## Dataset Structure72 73### Data Instances74 75One directory per fixture, named by its numeric id. There are two shapes:76 77```78# Generation fixture79<id>/80├── description.yaml      # prompt + metadata81└── input.png            # the engineering drawing (input2.png, ... when multi-image)82 83# Editing fixture84<id>/85├── description.yaml      # prompt + metadata86├── edit_description.txt  # the requested change, as an instruction87├── input.step           # the starting solid to edit88├── input.mesh.npz       # trusted watertight mesh sidecar for input.step89└── renders/             # iso / front / top / right PNGs of the starting solid90```91 92### Data Fields93 94`description.yaml` carries:95 96| Field | Type | Description |97|---|---|---|98| `description` | string | The task prompt. |99| `task_type` | `"generation"` \| `"editing"` | The task family. |100| `input_files` | list of strings | The input files the task declares (e.g. `input.png`, `input2.png`, or `input.step`). |101| `input_type` | `"text+image"` \| `"text+step"` | Modality of the inputs. |102 103### Data Splits104 105No train/test split — all 81 fixtures form a single evaluation set106(49 generation, 32 editing).107 108## Dataset Creation109 110### Source Data111 112The underlying CAD geometry is sourced from [Mecado](https://www.mecado.com).113Fixtures are real mechanical parts with mating interfaces (locating jigs, bolt114patterns, slots). For each part, an engineering drawing (generation) or a115starting solid plus an edit instruction (editing) forms the public input; the116solved solid and its interface sub-volumes are held privately as ground truth.117 118### Curation Rationale119 120The inputs are released publicly so contestants see exactly what they are121solving, while the ground truth stays private so the leaderboard's server-side122evaluation is the only path to a score.123 124## Considerations for Using the Data125 126This is a benchmark **input** set, not a training corpus. To participate,127generate one `output.step` per fixture and submit through the leaderboard128Space.129 130## Additional Information131 132### Dataset Curators133 134The CADGenBench team ([HuggingAI4Engineering](https://huggingface.co/HuggingAI4Engineering)).135 136### Licensing Information137 138Released under the [Open Data Commons Attribution License (ODC-BY)](https://opendatacommons.org/licenses/by/1-0/).139 140### Acknowledgements141 142CAD geometry sourced from [Mecado](https://www.mecado.com). Thanks to the143Mecado team.144