cmudrc/OpenSeeSimE-Fluid-Mini

OpenSeeSimE-Fluid-Mini

A stratified 1% subset of `cmudrc/OpenSeeSimE-Fluid` for evaluating vision-language models at a reduced compute footprint while preserving the joint distribution of simulation type, question type, media type, and question id.

Subset Provenance

• —Parent dataset: `cmudrc/OpenSeeSimE-Fluid` (98,326 rows total)
• —Rows in this subset: 1,040 (1.06% of parent)
• —Source classes: Bent Pipe, Converging Nozzle, Heat Exchanger, Heat Sink, Mixing Pipe
• —Parquet shards: 3 | Storage: ~11.61 GB
• —Sampling: per-stratum shuffle with numpy.random.default_rng(42), then take ceil(n * fraction) from each stratum. Any non-empty stratum contributes at least 1 row.
• —Strata: (source_file, question_type, media_type, question_id) — all four jointly.
• —Nesting: the 1% subset is a literal subset of the 10% subset (same shuffled prefix is taken for every fraction).

Composition

By source_file

By media_type

By (source_file, question_type)

Feature Schema

Identical to the parent dataset. See `cmudrc/OpenSeeSimE-Fluid` for full documentation of simulation generation, ground-truth extraction, preprocessing, limitations, and intended use.

{
    'file_name':          str,        # Unique identifier
    'source_file':        str,        # Base simulation model
    'question':           str,        # Question text
    'question_type':      str,        # 'Binary', 'Multiple Choice', 'Spatial'
    'question_id':        int,        # Question identifier (1-20)
    'answer':             str,        # Ground truth answer
    'answer_choices':     list[str],  # Options
    'correct_choice_idx': int,        # Index of correct answer
    'image':              Image,      # PIL Image (1920x1440) or null for video rows
    'video':              Video,      # Video bytes or null for image rows
    'media_type':         str,        # 'image' or 'video'
}

Intended Use

• —Benchmark evaluation of vision-language models on engineering simulation question answering at reduced compute cost
• —Smoke-testing of evaluation pipelines before running the full benchmark
• —Comparative studies where storage or bandwidth constraints matter

License

MIT — same as parent. Free for academic and commercial use with attribution.

Citation

@article{ezemba2024opensesime,
  title={OpenSeeSimE: A Large-Scale Benchmark to Assess Vision-Language Model Question Answering Capabilities in Engineering Simulations},
  author={Ezemba, Jessica and Pohl, Jason and Tucker, Conrad and McComb, Christopher},
  year={2025}
}

Contact

Jessica Ezemba — jezemba@andrew.cmu.edu Department of Mechanical Engineering, Carnegie Mellon University