Datasetpublic

rafmacalaba/data-use-ner

Data-use-ner (human holdout) GLiNER-format human-adjudicated holdout: 473 spans — annotator190 (190, origin=fcv_pads_east_africa) + jdc283 (283, origin=jdc_operational). Never trained on. Source: rafmacalaba/datause-displacement-reviewed holdout (gliner_reviewed token spans + readable_reviewed passages, v2.4 labels) with v3 probe head_score (outputs/gliner_datause_v3_probe_human473.jsonl). Columns text (full passage = " ".join(tokenized_text); span char offsets… See the full description on the dataset page: https://huggingface.co/datasets/rafmacalaba/data-use-ner.

sourceHugging Faceapache-2.0updated 12d agoView on Hugging Face

0likes214downloads

Dataset Card

Data-use-ner (human holdout)

GLiNER-format human-adjudicated holdout: 473 spans — annotator190 (190, origin=fcv_pads_east_africa) + jdc283 (283, origin=jdc_operational). Never trained on.

Source: rafmacalaba/datause-displacement-reviewed holdout (gliner_reviewed token spans + readable_reviewed passages, v2.4 labels) with v3 probe head_score (outputs/gliner_datause_v3_probe_human473.jsonl).

Columns

text (full passage = " ".join(tokenized_text); span char offsets index into it) + tokenized_text (whitespace tokens) + ner (catch-all DATA_MENTION word spans; drops are [] by convention — truth is spans[].luna_label) + traceability corpus_id, page, chunk, split, spans (per-span text, pred @ threshold 0.5, luna_label 1/0 (v2.4 gold), human_verdict agree/disagree/unsure + human_note (raw annotator input), head_score, threshold, char start/end into " ".join(tokenized_text), key, source).

Usage

python

from datasets import load_dataset
ds = load_dataset("rafmacalaba/data-use-ner", "gliner", split="holdout")  # 473 rows

python

ann = load_dataset("rafmacalaba/data-use-ner", "to_annotate")  # Luna-verdict spans + passages, human_verdict null