danish-foundation-models/dala

DaLA: Danish Linguistic Acceptability Evaluation Dataset

DaLA ([paper][1]) is a benchmark dataset for linguistic acceptability judgment in Danish, designed to evaluate how well NLP models, especially large language models (LLMs), understand grammaticality in real-world Danish sentences. The dataset extends previous resources by introducing a broader and more realistic set of error types and providing data splits suitable for evaluation via few-shot or finetuning.

🔗 Links

• —DaLA variants are linked and described below
• —[Paper][1]
• —GitHub Repository (code, data generation scripts)

📖 Overview

In linguistic acceptability tasks, models must distinguish between grammatically acceptable and unacceptable sentences. The DaLA dataset was created by:

• —Analyzing real-world Danish writing errors.
• —Designing 14 distinct corruption functions that reflect common Danish mistakes (e.g., pronoun confusion, suffix errors, interchange of determiners).
• —Applying these corruptions to correct Danish sentences from the Universal Dependencies Danish corpus.
• —Pairing each corrupted sentence with its correct counterpart.

The dataset includes:

• —The original correct sentences (acceptable).
• —The corrupted sentences (unacceptable).
• —A binary acceptability label.
• —A corruption type identifier.

📦 Dataset Variants and Splits

There are three variants of the DaLA dataset, each with different sizes and proportions:

Each variant includes train, validation, and test splits.

🧠 Tasks & Usage

DaLA is primarily intended for:

✔ Model evaluation and benchmarking: Assessing model competence in grammatical judgment ✔ Minimal-pair evaluation: Error type discrimination and fine-grained analysis

You can load the dataset using the Hugging Face datasets library as follows:

from datasets import load_dataset

# Standard split
dataset = load_dataset("giannor/dala")

# Medium or large variants
dataset_medium = load_dataset("giannor/dala_medium")
dataset_large  = load_dataset("giannor/dala_large")

📊 Baselines & Model Performance

In the corresponding paper, DaLA was used to benchmark a variety of open-source LLMs and model types. Across many models, performance on DaLA was lower than on previous Danish acceptability benchmarks, highlighting DaLA’s greater difficulty and discriminatory power. ([DaLA paper][1])

📄 Citation

If you use this dataset in your work, please cite the following paper:

@misc{barmina2025daladanishlinguisticacceptability,
      title={DaLA: Danish Linguistic Acceptability Evaluation Guided by Real World Errors}, 
      author={Gianluca Barmina and Nathalie Carmen Hau Norman and Peter Schneider-Kamp and Lukas Galke},
      year={2025},
      eprint={2512.04799},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2512.04799}, 
}

⚖️ License

This dataset is shared under the CC BY 4.0 license.

[1]: https://arxiv.org/abs/2512.04799 "DaLA: Danish Linguistic Acceptability Evaluation Guided by Real World Errors"