notefill/ck12-tqa-instruction

CK-12 TQA: Textbook Question Answering (Instruction Format)

Dataset Description

Dataset Summary

This is a reformatted version of the TQA (Textbook Question Answering) dataset, converted into an instruction-following format suitable for training and evaluating large language models on science question answering and multimodal reasoning tasks.

The TQA dataset consists of 1,076 lessons from Life Science, Earth Science, and Physical Science textbooks sourced from CK-12 Foundation. Each lesson contains multiple-choice questions addressing key concepts taught in that lesson. The dataset includes both text-only questions and questions requiring diagram comprehension.

Key Features:

• —📚 26,260 total questions from middle school science textbooks
• —🎯 Instruction-following format ready for LLM fine-tuning
• —🖼️ 12,567 diagram-based questions (multimodal reasoning)
• —📝 13,693 text-only questions
• —🔬 Covers Life Science, Earth Science, and Physical Science
• —📊 Train/Val/Test splits at lesson level to minimize concept overlap

Original Source

This dataset is derived from:

• —Original Paper: Are You Smarter Than A Sixth Grader? Textbook Question Answering for Multimodal Machine Comprehension (CVPR 2017)
• —Authors: Aniruddha Kembhavi, Minjoon Seo, Dustin Schwenk, Jonghyun Choi, Ali Farhadi, Hannaneh Hajishirzi
• —Content Source: CK-12 Foundation open-source science curriculum
• —License: Creative Commons Attribution-NonCommercial 3.0 Unported (CC BY-NC 3.0)

Supported Tasks

• —Instruction Following: Training models to answer textbook questions
• —Question Answering: Multiple-choice question answering
• —Visual Question Answering: Diagram-based question answering (multimodal)
• —Science Education: Middle school level science comprehension
• —Multimodal Reasoning: Combining text and visual information

Languages

The dataset is in English (en).

Dataset Structure

Data Format

Each example in the instruction format contains:

Non-Diagram Question Example:

{
  "id": "NDQ_000046",
  "source": "TQA-CK12",
  "split": "train",
  "lesson_id": "L_0002",
  "lesson_name": "earth science and its branches",
  "has_diagram": false,
  "instruction": "Answer the following multiple choice question from a science textbook.",
  "input": "Earth science is the study of\n\nOptions:\na) solid Earth.\nb) Earths oceans.\nc) Earths atmosphere.\nd) all of the above",
  "output": "d",
  "question_type": "Multiple Choice",
  "question_subtype": "Multiple Choice",
  "options": ["solid Earth.", "Earths oceans.", "Earths atmosphere.", "all of the above"],
  "option_labels": ["a", "b", "c", "d"]
}

Diagram Question Example:

{
  "id": "DQ_000001",
  "source": "TQA-CK12",
  "split": "train",
  "lesson_id": "L_0003",
  "lesson_name": "erosion and deposition by flowing water",
  "has_diagram": true,
  "instruction": "Answer the following multiple choice question about the diagram from a science textbook.",
  "input": "How many actions are depicted in the diagram?\n\nOptions:\na) 6\nb) 4\nc) 8\nd) 7",
  "output": "d",
  "question_type": "Diagram Multiple Choice",
  "question_subtype": "",
  "image_path": "question_images/erosion_6843.png",
  "options": ["6", "4", "8", "7"],
  "option_labels": ["a", "b", "c", "d"]
}

Data Fields

• —id: Unique question identifier (DQ* for diagram questions, NDQ* for non-diagram)
• —source: Dataset source ("TQA-CK12")
• —split: Data split ("train", "validation", or "test")
• —lesson_id: Identifier for the lesson containing this question
• —lesson_name: Name of the lesson topic
• —has_diagram: Boolean indicating if question requires a diagram
• —instruction: The instruction prompt for the model
• —input: The formatted question with answer options
• —output: The correct answer (letter label)
• —question_type: Type of question (Multiple Choice, True/False, etc.)
• —question_subtype: More specific question categorization
• —options: List of answer choice texts
• —option_labels: List of answer choice labels (a, b, c, d, etc.)
• —image_path: Relative path to diagram image (only for diagram questions)

Data Splits

The dataset is split at the lesson level to minimize concept overlap between splits:

Files

• —ck12_tqa_train.jsonl - Training set (15,154 questions)
• —ck12_tqa_val.jsonl - Validation set (5,309 questions)
• —ck12_tqa_test.jsonl - Test set (5,797 questions)

Note: This instruction-format version contains only the questions and answers. The full TQA dataset includes images, lesson content, instructional diagrams, and detailed annotations. For the complete dataset with images, please refer to the original TQA dataset.

Dataset Creation

Source Data Curation

The original TQA dataset was created from CK-12 Foundation's open-source science textbooks:

• —Subject Areas: Life Science, Earth Science, Physical Science
• —Target Audience: Middle school students (approximately grades 6-8)
• —Question Types: Multiple choice, True/False, Fill-in-the-blank
• —Number of Answer Choices: Varies from 2 to 7 options per question
• —Lesson Structure: Questions paired with instructional content from textbook lessons

Conversion Process

This instruction-format version was created by:

1. 1.Extraction: Extracted questions from lesson JSON structures
2. 2.Formatting: Converted to instruction-input-output format
3. 3.Categorization: Separated diagram and non-diagram questions
4. 4.Metadata Addition: Added lesson names, question IDs, and split information
5. 5.Standardization: Normalized question and answer text

The conversion preserves all original questions while making the format more suitable for modern instruction-tuned language models.

Subject Coverage

The dataset covers three main science domains:

Life Science

• —Cell biology and genetics
• —Ecology and ecosystems
• —Human body systems
• —Evolution and natural selection
• —Organisms and classification

Earth Science

• —Geology and Earth's structure
• —Weathering and erosion
• —Oceans and atmosphere
• —Climate and weather
• —Solar system and astronomy

Physical Science

• —Matter and its properties
• —Chemical reactions
• —Forces and motion
• —Energy and waves
• —Electricity and magnetism

Usage

Loading the Dataset

from datasets import load_dataset

# Load the full dataset
dataset = load_dataset("notefill/ck12-tqa-instruction")

# Load specific splits
train_data = load_dataset("notefill/ck12-tqa-instruction", data_files="ck12_tqa_train.jsonl")
val_data = load_dataset("notefill/ck12-tqa-instruction", data_files="ck12_tqa_val.jsonl")
test_data = load_dataset("notefill/ck12-tqa-instruction", data_files="ck12_tqa_test.jsonl")

Filter by Question Type

import json

# Load and filter non-diagram questions
non_diagram_questions = []
with open("ck12_tqa_train.jsonl", "r") as f:
    for line in f:
        q = json.loads(line)
        if not q["has_diagram"]:
            non_diagram_questions.append(q)

# Load and filter diagram questions
diagram_questions = []
with open("ck12_tqa_train.jsonl", "r") as f:
    for line in f:
        q = json.loads(line)
        if q["has_diagram"]:
            diagram_questions.append(q)

Fine-tuning Example

from datasets import load_dataset

# Load training data
dataset = load_dataset("notefill/ck12-tqa-instruction", data_files="ck12_tqa_train.jsonl")

# Format for instruction tuning
def format_prompt(example):
    return {
        "prompt": f"{example['instruction']}\n\n{example['input']}",
        "completion": example['output']
    }

formatted_dataset = dataset.map(format_prompt)

Evaluation Example

def evaluate_accuracy(predictions, dataset):
    """Calculate accuracy of predictions"""
    correct = 0
    total = 0
    
    for pred, item in zip(predictions, dataset):
        if pred.strip().lower() == item['output'].strip().lower():
            correct += 1
        total += 1
    
    return correct / total if total > 0 else 0.0

Considerations for Using the Data

Recommended Uses

✅ Training question-answering models for science education ✅ Evaluating multimodal reasoning capabilities ✅ Building educational AI tutoring systems ✅ Research in textbook comprehension ✅ Developing visual question answering models ✅ Benchmarking middle school science knowledge

Limitations

⚠️ Images Not Included: This instruction format version contains only text. For diagram questions, image paths are provided but images must be obtained from the original TQA dataset ⚠️ Answer Format: Only answer labels (a, b, c, d) are provided, not full answer text ⚠️ Grade Level: Limited to middle school science topics ⚠️ Language: English only ⚠️ Subject Coverage: Limited to Life, Earth, and Physical Science ⚠️ Question Types: Primarily multiple choice format

Ethical Considerations

• —Dataset designed for research and educational purposes
• —Content sourced from CK-12 Foundation's open educational resources
• —Non-commercial license restricts commercial applications
• —Human oversight recommended for student-facing applications
• —Care should be taken to ensure equitable access to educational AI tools

Benchmark Performance

From the original TQA paper (CVPR 2017):

The dataset was designed to be challenging for machine comprehension systems. Performance on diagram questions typically lags behind text-only questions, highlighting the difficulty of multimodal reasoning.

Citation

Original TQA Dataset

@inproceedings{Kembhavi2017tqa,
  title={Are You Smarter Than A Sixth Grader? Textbook Question Answering for Multimodal Machine Comprehension},
  author={Aniruddha Kembhavi and Minjoon Seo and Dustin Schwenk and Jonghyun Choi and Ali Farhadi and Hannaneh Hajishirzi},
  booktitle={Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2017}
}

This Instruction Format Version

@dataset{ck12_tqa_instruction2025,
  title={CK-12 TQA: Textbook Question Answering (Instruction Format)},
  author={Kuyeso Rogers and Adiza Alhassan and Notefill},
  year={2025},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/datasets/notefill/ck12-tqa-instruction}},
  note={Instruction-following format conversion of the TQA dataset from CK-12 textbooks}
}

Licensing Information

This dataset is distributed under the Creative Commons Attribution-NonCommercial 3.0 Unported (CC BY-NC 3.0) license, consistent with the original TQA dataset and CK-12 Foundation's licensing terms.

License Summary

• —✅ Share: Copy and redistribute the material in any medium or format
• —✅ Adapt: Remix, transform, and build upon the material
• —❌ NonCommercial: You may not use the material for commercial purposes
• —📝 Attribution: You must give appropriate credit to CK-12 Foundation and the TQA authors

For commercial use, please contact CK-12 Foundation directly.

Additional Resources

• —Original TQA Dataset: allenai.org/data/tqa
• —CK-12 Foundation: www.ck12.org
• —Original Paper: CVPR 2017 Paper
• —Full Dataset with Images: Available from Allen Institute for AI

Acknowledgments

We gratefully acknowledge:

• —CK-12 Foundation for creating and freely distributing high-quality science educational materials
• —Original TQA Authors: Aniruddha Kembhavi, Minjoon Seo, Dustin Schwenk, Jonghyun Choi, Ali Farhadi, and Hannaneh Hajishirzi for constructing this valuable dataset
• —Allen Institute for AI (AI2) for hosting and maintaining the original dataset
• —The many educators and content creators who contributed to the CK-12 curriculum

About CK-12 Foundation

The CK-12 Foundation is a non-profit organization dedicated to increasing access to high-quality K-12 STEM education. They provide free, openly-licensed educational content that can be customized to meet the needs of teachers and students worldwide.

Contact

For questions or feedback about this instruction-format version, please open an issue on the dataset repository.

For questions about the original TQA dataset, please refer to the Allen Institute for AI.

For information about CK-12 content and licensing, visit www.ck12.org.