CoolFace
Datasetpublic

upb-nlp/EduMUSE

OpenStax Multimodal Exercise Dataset A multimodal dataset of textbook exercises scraped from OpenStax, each aligned to its most relevant textbook subsection and scored under several open vision-language models. The dataset enables research on retrieval-augmented question answering, the contribution of visual context to scientific QA, and ablation studies on text-only vs. multimodal context. Contents final_EduMUSE_dataset.json — the unified dataset (nested by book… See the full description on the dataset page: https://huggingface.co/datasets/upb-nlp/EduMUSE.

sourceHugging Facecc-by-4.0updated 5mo agoView on Hugging Face
1likes112downloads
Dataset Card

OpenStax Multimodal Exercise Dataset

A multimodal dataset of textbook exercises scraped from OpenStax, each aligned to its most relevant textbook subsection and scored under several open vision-language models. The dataset enables research on retrieval-augmented question answering, the contribution of visual context to scientific QA, and ablation studies on text-only vs. multimodal context.

Contents

  • —final_EduMUSE_dataset.json — the unified dataset (nested by book → chapter → exercise).
  • —context_images.zip — images extracted from textbook sections (extract to context_images/).
  • —exercise_images.zip — images attached to individual exercises (extract to exercise_images/).

Dataset structure

Top-level JSON shape:

json
{
  "<book_name>": {
    "chapters": {
      "<chapter_id>": {
        "sections": [
          {
            "section_name": "...",
            "content": [
              {"items": [{"type": "text|list_item|image", "content"|"filename"|"alt_text": "..."}]}
            ]
          }
        ],
        "exercises": [
          {
            "exercise_id": "...",
            "problem_text": "...",
            "solution_text": "...",
            "has_solution": true,
            "exercise_images": [{"filename": "..."}],
            "best_subsection_data": {
              "best_section_name": "...",
              "best_subsection_index": 0
            }
          }
        ]
      }
    }
  }
}

Loading

This dataset is published as raw files (JSON + image folders), so download the whole snapshot and load locally:

python
from huggingface_hub import snapshot_download
import json
import zipfile
from PIL import Image
from pathlib import Path

local_dir = Path(snapshot_download(
    repo_id="your-username/openstax-multimodal-exercises",
    repo_type="dataset",
))

# Extract the image archives once; skip if already extracted.
for archive in ["context_images.zip", "exercise_images.zip"]:
    target = local_dir / archive.removesuffix(".zip")
    if not target.exists():
        with zipfile.ZipFile(local_dir / archive) as zf:
            zf.extractall(local_dir)

with open(local_dir / "final_EduMUSE_dataset.json") as f:
    data = json.load(f)

# Iterate exercises with their matched subsection
for book_name, book in data.items():
    for chapter_id, chapter in book["chapters"].items():
        for exercise in chapter["exercises"]:
            if "best_subsection_data" not in exercise:
                continue
            for img_ref in exercise.get("exercise_images", []):
                fname = img_ref["filename"].split("_")[-1] + ".jpg"
                img = Image.open(local_dir / "exercise_images" / fname)
                # ... do something with img + exercise["problem_text"]

Source and licensing

Source content is from OpenStax textbooks, released under the Creative Commons Attribution 4.0 International License (CC BY 4.0). This derivative dataset is distributed under the same license. When using this dataset you must:

  • —Attribute OpenStax for the underlying textbook content.
  • —Attribute this dataset for the multimodal alignment.

Specific OpenStax titles included are identifiable from the <book_name> keys at the top level of the JSON.

Known limitations

  • —chemistry-atoms-first-2e chapter 8 is excluded due to corrupted source data.
  • —Subsection alignment is computed via a single VLM (Qwen2-VL-7B) and is not human-verified — it is the model's minimum-loss pick, not a ground-truth label.
  • —Image content is filtered to JPEGs scraped at the time the pipeline was run; some original OpenStax figures may have changed since.

Citation

If you use this dataset, please cite:

bibtex
@misc{openstax-multimodal-exercises,
  title  = {OpenStax Multimodal Exercise Dataset},
  author = {<your name(s) here>},
  year   = {2026},
  url    = {https://huggingface.co/datasets/your-username/openstax-multimodal-exercises}
}

And cite OpenStax for the source content per their attribution guidelines.