oliverkinch/da-instruct-dynaword

da-instruct-dynaword

Danish instruction fine-tuning dataset generated via backtranslation from `danish-foundation-models/danish-dynaword`, filtered to high-quality samples using `danish-foundation-models/dynaword-annotations`.

Dataset description

Each row is a (prompt, target) pair where:

• —`target` is a passage of authentic Danish text drawn from a curated subset of DynaWord
• —`prompt` is a realistic Danish user instruction that would plausibly elicit that text from a language model

Prompts were generated by Qwen/Qwen3.5-397B-A17B using a backtranslation approach: given a passage and its source context, the model is asked to write the user message that would cause a chatbot to produce a similar text. Domain-specific prompt templates (government, literary, academic, tax guidance, news, speech) ensure that the generated instructions match the register and genre of the source text.

Construction

1 — Quality filtering with dynaword-annotations

The dynaword-annotations dataset provides per-sample quality annotations for the full 5.66M-sample DynaWord corpus. The following filter was applied:

This yields 233,570 candidate IDs (4.1% of the corpus). To balance coverage across domains, a maximum of 2,000 IDs per dynaword subset was sampled, giving 36,352 source IDs across 40 subsets.

2 — Passage extraction

Articles shorter than 400 characters were discarded. Longer articles were chunked into passages of up to 2,500 characters using a hybrid scoring approach (paragraph boundary detection with quality scoring). A maximum of 2 passages per article was kept.

Passage-level quality filters reject:

• —OCR noise (inter-word double spaces, ligature splits, high mixed-alphanumeric token ratio)
• —Repetitive content (unique word ratio < 28% or word run ≥ 6)
• —Document colophons and mastheads (phone/email/ISSN markers in first 300 chars)
• —YAML/markdown frontmatter

3 — Prompt generation

For each passage, Qwen3.5-397B-A17B was prompted with the passage text and source context metadata to generate a Danish user instruction. Six domain-specific system prompts were used:

• —government — administrative and public information requests
• —literary — prose, fiction, letters and historical texts
• —academic — scholarly articles and analytical texts
• —tax_guidance — practical citizen queries about tax rules
• —speech — parliamentary speeches and public addresses
• —news — news articles and encyclopedic texts (default)

Generated prompts were validated with rule-based filters: language detection (must be Danish), user-request detection (must include a question or imperative verb), shouting token check, roleplay pattern rejection, and prompt-target content overlap check.

4 — Results

39,847 instruction pairs generated from 36,352 high-quality source IDs across 40 dynaword subsets.

Source coverage

Schema

{
    "id": "backtranslation_passages_dynaword-<hash>",
    "prompt": "Danish user instruction (generated)",
    "target": "Danish text passage (from source corpus)",
    "meta": {
        "passage_idx": 0,                # passage index within the source article
        "source_config_name": "skat",    # HuggingFace config name
        "source_dataset": "danish-foundation-models/danish-dynaword",
        "source_id": "skat::SKM...",     # original article ID
        "source_key": "skat",
        "source_name": "skat",           # dynaword subset name
        "source_record_index": 42,
        "source_split": "train",
        "source_type": "tax_guidance",   # prompt template used
        "target_chars": 1842
    },
    "sources": [
        {"config_name": "skat", "dataset": "danish-foundation-models/danish-dynaword",
         "row_id": "skat::SKM...", "split": "train"}
    ]
}

Usage

from datasets import load_dataset

ds = load_dataset("oliverkinch/da-instruct-dynaword")
# ds["train"], ds["test"]

# Format for SFT
def format_example(row):
    return {
        "messages": [
            {"role": "user",      "content": row["prompt"]},
            {"role": "assistant", "content": row["target"]},
        ]
    }

Licence

This dataset inherits the licences of the constituent DynaWord subsets. Subsets carry a range of open licences (CC0, CC-BY, CC-BY-SA, government open data). See the DynaWord dataset card for per-subset licence details.

The generated prompt column was produced with Qwen/Qwen3.5-397B-A17B and is released under CC-BY 4.0.

Related datasets

• —`danish-foundation-models/danish-dynaword` — source corpus
• —`danish-foundation-models/dynaword-annotations` — quality annotations used for filtering