CoolFace
Datasetpublic

HAERAE-HUB/HAE_RAE_BENCH_1.1

The HAE_RAE_BENCH 1.1 is an ongoing project to develop a suite of evaluation tasks designed to test the understanding of models regarding Korean cultural and contextual nuances. Currently, it comprises 13 distinct tasks, with a total of 4900 instances. Please note that although this repository contains datasets from the original HAE-RAE BENCH paper, the contents are not completely identical. Specifically, the reading comprehension subset from the original version has been removed due to… See the full description on the dataset page: https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1.

sourceHugging Facecc-by-nc-nd-4.0updated 2y agoView on Hugging Face
20likes3.4kdownloads
Dataset Card

The HAERAEBENCH 1.1 is an ongoing project to develop a suite of evaluation tasks designed to test the understanding of models regarding Korean cultural and contextual nuances. Currently, it comprises 13 distinct tasks, with a total of 4900 instances.

Please note that although this repository contains datasets from the original HAE-RAE BENCH paper, the contents are not completely identical. Specifically, the reading comprehension subset from the original version has been removed due to copyright constraints. In its place, an updated reading comprehension subset has been introduced, sourced from the CSAT, the Korean university entrance examination. To replicate the studies from the paper, please see code.

Dataset Overview

TaskInstancesVersionExplanation
standard_nomenclature153v1.0Multiple-choice questions about Korean standard nomenclatures from NIKL.
loan_word169v1.0Multiple-choice questions about Korean loan words from NIKL.
rare_word405v1.0Multiple-choice questions about rare Korean words from NIKL.
general_knowledge176v1.0Multiple-choice questions on Korean cultural knowledge.
history188v1.0Multiple-choice questions on Korean history.
date_understanding475v1.1Multiple-choice questions about understanding Korean date expressions.
proverbs_denoising672v1.1Questions for generating Korean proverbs.
correctdefinitionmatching439v1.1Multiple-choice questions on matching the correct definition to words in context.
lyrics_denoising622v1.1Questions for reconstructing Korean pop song lyrics.
reading_comprehension936v1.1Multiple-choice questions on Korean reading comprehension from the CSAT.
csat_law217v1.1Multiple-choice questions on Korean law from the CSAT.
csat_geo150v1.1Multiple-choice questions on Korean geography from the CSAT.
csat_socio298v1.1Multiple-choice questions on Korean social studies from the CSAT.
Total4900

Point of Contact

For any questions contact us via the following email:)

spthsrbwls123@yonsei.ac.kr