cmudrc/OpenSeeSimE-Fluid-Small

1---2dataset_info:3  features:4  - name: file_name5    dtype: string6  - name: source_file7    dtype: string8  - name: question9    dtype: string10  - name: question_type11    dtype: string12  - name: question_id13    dtype: int3214  - name: answer15    dtype: string16  - name: answer_choices17    list: string18  - name: correct_choice_idx19    dtype: int3220  - name: image21    dtype: image22  - name: video23    dtype: video24  - name: media_type25    dtype: string26  splits:27  - name: test28    num_examples: 988129configs:30- config_name: default31  data_files:32  - split: test33    path: data/test-*34license: mit35task_categories:36- visual-question-answering37language:38- en39size_categories:40- 1K<n<10K41tags:42- engineering43- simulation44- stratified-subset45---46 47# OpenSeeSimE-Fluid-Small48 49A **stratified 10% subset** of [`cmudrc/OpenSeeSimE-Fluid`](https://huggingface.co/datasets/cmudrc/OpenSeeSimE-Fluid) for evaluating vision-language models at a reduced compute footprint while preserving the joint distribution of simulation type, question type, media type, and question id.50 51## Subset Provenance52 53- **Parent dataset**: [`cmudrc/OpenSeeSimE-Fluid`](https://huggingface.co/datasets/cmudrc/OpenSeeSimE-Fluid) (98,326 rows total)54- **Rows in this subset**: **9,881** (10.05% of parent)55- **Source classes**: `Bent Pipe`, `Converging Nozzle`, `Heat Exchanger`, `Heat Sink`, `Mixing Pipe`56- **Parquet shards**: 19   |   **Storage**: ~103.68 GB57- **Sampling**: per-stratum shuffle with `numpy.random.default_rng(42)`, then take `ceil(n * fraction)` from each stratum. Any non-empty stratum contributes at least 1 row.58- **Strata**: `(source_file, question_type, media_type, question_id)` — all four jointly.59- **Nesting**: the 1% subset is a literal subset of the 10% subset (same shuffled prefix is taken for every fraction).60 61## Composition62 63### By `source_file`64 65| source_file       |   rows |   pct |66|:------------------|-------:|------:|67| Mixing Pipe       |   2070 | 20.95 |68| Heat Exchanger    |   2029 | 20.53 |69| Bent Pipe         |   1976 | 20.00 |70| Converging Nozzle |   1971 | 19.95 |71| Heat Sink         |   1835 | 18.57 |72 73### By `media_type`74 75| media_type   |   rows |76|:-------------|-------:|77| image        |   4948 |78| video        |   4933 |79 80### By `(source_file, question_type)`81 82| source_file       |   Binary |   Multiple Choice |   Spatial |   Total |83|:------------------|---------:|------------------:|----------:|--------:|84| Bent Pipe         |      792 |               796 |       388 |    1976 |85| Converging Nozzle |      791 |               789 |       391 |    1971 |86| Heat Exchanger    |      812 |               811 |       406 |    2029 |87| Heat Sink         |      719 |               710 |       406 |    1835 |88| Mixing Pipe       |      828 |               828 |       414 |    2070 |89 90## Feature Schema91 92Identical to the parent dataset. See [`cmudrc/OpenSeeSimE-Fluid`](https://huggingface.co/datasets/cmudrc/OpenSeeSimE-Fluid) for full documentation of simulation generation, ground-truth extraction, preprocessing, limitations, and intended use.93 94```python95{96    'file_name':          str,        # Unique identifier97    'source_file':        str,        # Base simulation model98    'question':           str,        # Question text99    'question_type':      str,        # 'Binary', 'Multiple Choice', 'Spatial'100    'question_id':        int,        # Question identifier (1-20)101    'answer':             str,        # Ground truth answer102    'answer_choices':     list[str],  # Options103    'correct_choice_idx': int,        # Index of correct answer104    'image':              Image,      # PIL Image (1920x1440) or null for video rows105    'video':              Video,      # Video bytes or null for image rows106    'media_type':         str,        # 'image' or 'video'107}108```109 110## Intended Use111 112- Benchmark evaluation of vision-language models on engineering simulation question answering at reduced compute cost113- Smoke-testing of evaluation pipelines before running the full benchmark114- Comparative studies where storage or bandwidth constraints matter115 116## License117 118MIT — same as parent. Free for academic and commercial use with attribution.119 120## Citation121 122```bibtex123@article{ezemba2024opensesime,124  title={OpenSeeSimE: A Large-Scale Benchmark to Assess Vision-Language Model Question Answering Capabilities in Engineering Simulations},125  author={Ezemba, Jessica and Pohl, Jason and Tucker, Conrad and McComb, Christopher},126  year={2025}127}128```129 130## Contact131 132**Jessica Ezemba** — jezemba@andrew.cmu.edu  133Department of Mechanical Engineering, Carnegie Mellon University134