jonasaise/swesat-skolprov-merged

SweSAT + Swedish Skolprov (Merged Dataset)

Dataset Description

This dataset is a unified, state-of-the-art benchmark designed for evaluating Large Language Models (LLMs) on Swedish text comprehension, vocabulary, and logical reasoning. It is constructed by merging two prominent Swedish test datasets:

1. 1.SweSAT-1.0: Questions sourced from the Swedish University Entrance Exam (Högskoleprovet) spanning from 2020-10-25 to 2024-04-13.
2. 2.Swedish Skolprov: Diverse Swedish academic and professional exams (e.g., apothecary, dentistry, math/physics).

Key Enhancements

• —LLM Evaluation Ready: Every question has been dynamically enriched with explicit system_prompt and prompt fields. These zero-shot instruction strings correspond perfectly with the formatting expected by modern LLM evaluation harnesses (such as lm-evaluation-harness).
• —Strict Quality Control: All visual-dependent questions and questions lacking pure-text multiple-choice options have been systematically filtered out.
• —Unified Schema: Both underlying datasets have been mapped to an identical structural format for seamless concatenation.

Dataset Structure

Each row represents a single multiple-choice question.

Data Fields

• —uid: A unique identifier for the question.
• —test_id: The date or ID of the specific exam/test.
• —section: The broader section of the exam.
• —subsection: The specific sub-category or question type (e.g., ORD, LÄS, MEK).
• —question_id: The numerical ID or specific question number from the source exam.
• —question_resource: Additional context or reading material required to answer the question (if applicable).
• —question: The actual text of the question.
• —option_a through option_e: The multiple-choice options (empty strings if the option does not exist).
• —system_prompt: A structured system prompt instructing the LLM on how to behave and format its output.
• —prompt: The user-facing prompt containing the exam instructions, the question, and the formatted choices.
• —answer: The correct answer key (e.g., "A", "B", "C", "D", "E").
• —source: Indicates the origin of the question ("swesat" or "skolprov").

Usage

This dataset is stored in the highly optimized Parquet format. You can load it directly via the Hugging Face datasets library:

from datasets import load_dataset

dataset = load_dataset("jonasaise/swesat-skolprov-merged")
print(dataset['train'][0])

@article{SweSAT2024,
  title={SweSAT-1.0: The Swedish University Entrance Exam as a Benchmark for Large Language Models},
  author={Kurfalı, Murathan and Zahra, Shorouq and Gogoulou, Evangelia and Dürlich, Luise and Carlsson, Fredrik and Nivre, Joakim},
  booktitle = "Proceedings of The Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025)",
  month = march,
  year = "2025",
  address = "Talinn, Estonia"
}