juletxara/lindsea-blimp

LINDSEA BLIMP

Dataset Description

LINDSEA BLIMP is a dataset of Indonesian linguistic minimal pairs for evaluating language models' syntactic knowledge. The dataset is based on the BHASA project's Indonesian syntax data.

Dataset Structure

The dataset contains minimal pairs of grammatical and ungrammatical sentences in Indonesian, organized into separate subsets testing various linguistic phenomena:

• —argument_structure (160 pairs): Tests word order, modals, and argument realization
• —npis_and_negation (20 pairs): Tests negative polarity items and negation licensing
• —filler_gap_dependencies (60 pairs): Tests relative clauses and wh-movement
• —morphology (140 pairs): Tests morphological constraints and word formation

Total: 380 minimal pairs across 4 subsets

Fields

• —sentence_good: The grammatical sentence
• —sentence_bad: The ungrammatical sentence
• —linguistic_phenomenon: The linguistic phenomenon being tested
• —category: The specific category within the phenomenon
• —subcategory: Further subcategorization (mostly "none")
• —original_id: Original ID from the source file

Usage

from datasets import load_dataset

# Load specific subset
dataset = load_dataset("juletxara/lindsea-blimp", "argument_structure")

# Or load all subsets
dataset = load_dataset("juletxara/lindsea-blimp")

# Access specific subset
arg_structure_data = dataset['argument_structure']

# Example usage
for example in arg_structure_data:
    print(f"Good: {example['sentence_good']}")
    print(f"Bad: {example['sentence_bad']}")
    print(f"Phenomenon: {example['linguistic_phenomenon']}")
    print()

Data Source

This dataset is derived from the BHASA project's Indonesian syntax minimal pairs: https://github.com/aisingapore/BHASA/tree/main/lindsea/id/syntax

Citation

Please cite the original BHASA paper when using this dataset:

@article{leong2023bhasa,
  title={BHASA: A Holistic Southeast Asian Linguistic Resource},
  author={Leong, Wei Qin and Kandpal, Nikhil and Kew, Tannon and Dabre, Raj and Aji, Alham Fikri and Winata, Genta Indra and Rocha, Samuel and Dou, Zheng-Xin and Freitag, Markus and Xu, Dan},
  journal={arXiv preprint arXiv:2309.06085},
  year={2023}
}

License

The dataset follows the original BHASA project's licensing terms.