NbAiLab/nb-fineweb2-edu-bokmaal

NB-Fineweb2-Edu-Bokmaal

A filtered and annotated subset of the Norwegian nb-fineweb2-c-edu-nob dataset, with added educational value scores derived from human ratings.

Dataset Summary

This dataset includes Norwegian-language web texts that have been annotated for educational value by human reviewers. It is designed to support research on text usefulness for educational applications such as language learning, tutoring, and classroom use.

The final dataset includes:

• —Texts that are mostly or partially in Norwegian
• —High-confidence language predictions (metadata.language_score > 0.99)
• —Aggregated educational value scores, flattened from multiple annotator responses

Construction Process

Starting Point

We began with a raw dataset containing human annotations under the Educational Value.responses column, alongside metadata such as predicted language and text ID.

Educational Value Scoring

Each example could have multiple human responses rating the educational value of the text. These responses were mapped to numeric scores using the following scale:

• —❗ Problematic Content ❗ = 0
• —None = 0
• —Minimal = 1
• —Basic = 2
• —Good = 3
• —Excellent = 4

To flatten multiple responses into a single score, the maximum value was selected when in disagreement. This ensures that even if some annotators rated a text poorly, any strong positive signal is preserved for downstream training or analysis.

def flatten_educational_value(responses_list):
    rank = {
        "❗ Problematic Content ❗": 0,
        "None": 0,
        "Minimal": 1,
        "Basic": 2,
        "Good": 3,
        "Excellent": 5
    }

    if responses_list is None or len(responses_list) == 0:
        return "None"

    cleaned = [rank[str(r).strip()] for r in responses_list if str(r).strip() in rank]
    return max(cleaned)

Filtering for Quality

The dataset was filtered to retain only high-confidence, relevant entries:

ddf = df[
    df["Correct language predicted.responses"].apply(
        lambda x: ("Mostly in-language" in x or "Small part in-language" in x)
    )
]
ddf = ddf[ddf["metadata.language_score"] > 0.99]

Features

Intended Use

• —Pretraining or finetuning educational models
• —Evaluating readability and pedagogical suitability of text
• —Norwegian NLP tasks related to education and curriculum design

License

Please refer to the original licensing terms of the base dataset. For derivative labeling and structure, this dataset is distributed under CC BY 4.0, unless noted otherwise.

Credits

This dataset was built from the original annotations made by the National Library of Norway Språkbanken and Språkmodeller units, with help from Daniel van Strien from Hugging Face.

Citation

If you use this dataset, please cite:

@misc{nb-fineweb2-edu-bokmaal,
  title={nb-fineweb2-edu-bokmaal},
  author={National Library of Norway},
  howpublished={\url{https://huggingface.co/datasets/NbAiLab/nb-fineweb2-edu-bokmaal}},
  year={2025}
}