shinysup/JBE-MC-original-format

Japanese Bar Examination Dataset (Original Format)

This dataset contains multiple-choice questions from the Japanese Bar Examination (司法試験 短答式), preserved in their original composite format and evaluated using the official grading scheme. This dataset has been constructed for the paper "Self-Verification is All You Need To Pass the Japanese Bar Examination"

Unlike prior datasets that decompose questions into independent true/false propositions, this dataset maintains the original structure where multiple statements must be jointly evaluated to produce a single composite answer (e.g., "11221").

Dataset Motivation

Recent work has reported strong performance on the Japanese Bar Examination using decomposed proposition-level datasets. However, such formulations differ substantially from the actual exam format and grading scale.

This dataset was constructed to enable:

• —Evaluation under authentic exam conditions
• —Measurement on the official 175-point scale
• —Analysis of composite legal reasoning across tightly coupled propositions

Dataset Structure

Each example includes:

Statistics

• —Total questions: 460
• —Format: Composite multi-proposition
• —Evaluation: Official Japanese bar exam grading scheme
• —Languages: Japanese

Usage

from datasets import load_dataset

dataset = load_dataset("shinysup/JBE-MC-original-format")
train = dataset["train"]
test = dataset["test"]