cogbench-anonymous/cogbench-passages

CogBench Passages

A curated collection of 120 short academic passages across 8 subjects, used as the source text for the CogBench benchmark — an evaluation of whether LLMs can generate questions that satisfy specified Bloom's-Taxonomy cognitive levels under deterministic, code-checkable constraints.

Anonymized for double-blind review (NeurIPS 2026 Evaluations & Datasets Track). Author and institutional metadata will be added after acceptance.

Evaluative role

This dataset is not a benchmark on its own. It is the source-text input for the CogBench evaluation pipeline, which:

1. 1.Asks an LLM to generate questions about a passage at a target Bloom's level (Remember / Understand / Apply / Analyze / Evaluate / Create).
2. 2.Runs 28 deterministic constraints (no LLM-as-judge) over the question + passage to produce a per-question constraint-satisfaction rate (CSR).
3. 3.Aggregates CSR across passages, subjects, and Bloom's levels.

The dataset is what lets the constraints reference real key_concepts and methods_principles so that constraints like "the question must reference a key concept from the passage" become decidable.

Schema

Quick load

from datasets import load_dataset

ds = load_dataset("cogbench-anonymous/cogbench-passages", split="train")
print(ds[0]["passage_id"], ds[0]["subject"], ds[0]["word_count"])

Composition

• —Total: 120 passages
• —Per subject: 15 each across 8 subjects
• —Source: OpenStax open-access textbooks (CC BY 4.0)

Collection process

• —Each subject's canonical OpenStax book was parsed; 15 sections per subject were sampled by length + topical-coverage heuristics.
• —Sections shorter than ~200 words or that were exercise/review blocks were dropped.
• —key_concepts and methods_principles were extracted with spaCy NER + noun-phrase mining.
• —Authors spot-checked passages for math/equation rendering artifacts; passages with unrecoverable LaTeX fragments were dropped.
• —Collection window: February–March 2026.

See the full datasheet (Gebru et al. 2021 format) in the source repository: proposal/methods/passages_datasheet.md.

Intended use

• —In scope: evaluating Bloom's-Taxonomy-aligned question generation by LLMs.
• —Out of scope: reading-comprehension benchmarks, fact-verification corpora, curriculum design, training data for instruction-tuned models.

Limitations

• —English-only. Non-English educational content is not represented.
• —120 passages is small; results should be reported with confidence intervals.
• —STEM-skewed — 6 of 8 subjects are STEM. Models tuned for STEM may have an unfair aggregate advantage; subject-level results are reported separately.
• —OpenStax-derived. Passages are likely present in many LLM pretraining corpora; CogBench reports a contamination probe alongside main results to quantify exposure.
• —Deterministic constraints as proxy. The 28 constraints encode common signals of cognitive level (verb choice, named-entity grounding, structural form) but do not directly measure whether a human would judge a question as "Analyze-level". This proxy assumption is part of what CogBench evaluates, not a guarantee.
• —Auto-extracted features. key_concepts and methods_principles are heuristic; they reflect what an automatic extractor surfaces, not a curated knowledge graph.

Ethical considerations

• —No personal or sensitive data; all content is pedagogical material already on the open web under CC BY 4.0.
• —No annotator labels — passages are reused verbatim.

Licensing

Attribution: OpenStax. Textbooks released under Creative Commons Attribution License 4.0. <https://openstax.org>

Versioning & maintenance

• —This is v1.0 (NeurIPS 2026 review).
• —A frozen DOI will be minted on Zenodo at camera-ready.
• —Future revisions will follow OpenStax book updates (e.g. Biology 2e → 3e); each release will keep prior versions identifiable in the changelog.

Citation

@misc{cogbench2026,
  title  = {CogBench: Deterministic Evaluation of Bloom's-Taxonomy-Aligned Question Generation by LLMs},
  author = {Anonymous},
  year   = {2026},
  note   = {NeurIPS 2026 Evaluations \& Datasets Track submission},
  url    = {https://huggingface.co/datasets/cogbench-anonymous/cogbench-passages}
}

Croissant metadata

The dataset includes a Croissant 1.0 metadata file (croissant.json) with both core and Responsible AI (RAI) fields, as required by the NeurIPS 2026 E&D Track.