jensjepsen/danish-extraction-v1

danish-extraction-v1

Danish information-extraction rows over real prose, where the schema is proposed per passage rather than fixed. Built from `danish-foundation-models/danish-dynaword` by scripts/gen_extraction_da.py.

Each source passage got its own field set: an LLM proposed 3-6 fields for that text without seeing any values, then filled them in a separate turn. Roughly a quarter of proposed fields come back empty, which are genuine abstention targets rather than annotation gaps.

Registers

Tasks

extract — given a passage and field names, produce each field's value. Prompts carrying an instruction also state the required output format, rendered from the format itself (Svar i formatet: felt værdi). An earlier release named no format in instruction-only prompts, which left one format out of ten unspecified and the row unanswerable.

fill — placement. The passage has gaps, the removed spans are listed in shuffled order, and the model reconstructs the text. Fully determined by the prompt: the information is all present and the task is to work out where each piece belongs. Scored on whether the spans land in the right sequence and the surrounding prose is reproduced, so neither echoing the gapped text nor echoing the value list earns credit.

An earlier release masked spans and asked the model to recall them, under a rule that a span occur exactly once so no copy stayed visible. Those two requirements conflict: "occurs exactly once" means the text holds no evidence for what was removed, and the masked spans are extraction values — the tokens least predictable from context. Probed, the model returned the right marker set, the right count and a clean parse, and scored 0 on every row, supplying a different plausible item from the same list.

What varies

Demonstrations carry their own text and their own field list, so they teach the task and the output format rather than a particular schema.

Gates

Every value is a verbatim span of its passage (up to whitespace), contains no newline, matches its declared type, and is deduplicated. Every rendered answer round-trips through the format parser.

Splits

eval_schema and eval_passage are 5% hash partitions on schema and on passage; eval_both is their intersection.

Configs

default (messages + meta), sft (messages only), and raw — the extractions the rows are rendered from, so formats, modes, gates and splits can be re-derived without re-extracting.

Generated with --registers --uniform, --rows-per-passage 4.