juletxara/blimp-nl

BLiMP-NL: Dutch BLIMP

Dataset Description

BLiMP-NL is a dataset of Dutch linguistic minimal pairs for evaluating language models' syntactic knowledge. It contains minimal pairs for 22 grammatical phenomena in Dutch, further divided into 84 paradigms.

Dataset Structure

The dataset contains minimal pairs of grammatical and ungrammatical sentences in Dutch, organized into subsets testing various linguistic phenomena. Each minimal pair tests a specific grammatical phenomenon.

Fields

• —sentence_good: The grammatical sentence
• —sentence_bad: The ungrammatical sentence
• —linguistic_phenomenon: The linguistic phenomenon being tested
• —paradigm: The specific paradigm within the phenomenon
• —item_nr: Item number that groups the minimal pair

Usage

from datasets import load_dataset

# Load specific subset
dataset = load_dataset("juletxara/blimp-nl", "agreement")

# Or load all subsets
dataset = load_dataset("juletxara/blimp-nl")

# Example usage
for subset_name, subset_data in dataset.items():
    print(f"Subset: {subset_name}")
    for example in subset_data:
        print(f"Good: {example['sentence_good']}")
        print(f"Bad: {example['sentence_bad']}")
        print(f"Phenomenon: {example['linguistic_phenomenon']}")
        print()

Data Source

This dataset is derived from BLiMP-NL: https://data.ru.nl/collections/ru/cls/blimp-nldsc550

Citation

Please cite the original BLiMP-NL paper when using this dataset:

@article{suijkerbuijk2023blimp,
  title={BLiMP-NL: The Benchmark of Linguistic Minimal Pairs for Dutch},
  author={Suijkerbuijk, M.J.P.F. and Prins, Zoë and de Heer Kloots, Marianne and Zuidema, Willem and Frank, Stefan},
  year={2023}
}

License

The dataset follows the original BLiMP-NL project's licensing terms.