Modelpublic

rafmacalaba/gliner-datause-probe

sourceHugging Faceapache-2.0updated 24d agoView on Hugging Face

0likes35downloads

Model Card

gliner-datause-probe

Fine-tune of urchade/gliner_large-v2.1 for data-use mention extraction with a single `DATA_MENTION` class, trained on `rafmacalaba/data-use-mentions-tiered` — the tiered copy of rafmacalaba/data-use-mentions where Luna/classifier-judged T3 (non-mention) and junk spans are untagged hard negatives (text stays, span removed). The extractor owns the mention boundary only (T1 evidential ∪ T2 declaration vs T3/junk); specificity detail is recovered downstream by the multitask SFT model.

Labels

—DATA_MENTION — a real data mention that carries an analytic or declarative use (T1 evidential ∪ T2 declaration)

Training

—base model: urchade/gliner_large-v2.1
—dataset: rafmacalaba/data-use-mentions-tiered (gliner_tiered config)
—epochs: 5
—learning rate: 5e-06
—batch size: 16
—precision: bf16
—checkpoint selection: val span-F0.5 (post-hoc sweep of epoch checkpoints; eval_loss was explicitly not used)

Evaluation (tiered holdout)

Gold = T1∪T2 spans; a true-FP cluster matching a dropped T3/junk span counts as a T3 leak (lower is better). Label-agnostic Hungarian matching, jaccard >= 0.5 — identical to prior data-use-mentions evals.

thr	tp	fp	fn	precision	recall	f0.5	f1	t3_leak	t3_leak%
0.10	16812	9110	353	0.6486	0.9794	0.6956	0.7804	2829	31.1%
0.20	16664	6850	501	0.7087	0.9708	0.7491	0.8193	2335	34.1%
0.30	16505	5607	660	0.7464	0.9615	0.7814	0.8404	2002	35.7%
0.40	16275	4623	890	0.7788	0.9482	0.8076	0.8552	1741	37.7%
0.50	15934	3701	1231	0.8115	0.9283	0.8325	0.8660	1498	40.5%
0.60	15195	2735	1970	0.8475	0.8852	0.8548	0.8659	1203	44.0%
0.70	13793	1808	3372	0.8841	0.8036	0.8667	0.8419	876	48.4%

Best F0.5: 0.8667 (thr=0.7) Best F1: 0.8660 (thr=0.5)

Corpus breakdown (holdout, best F0.5)

corpus	examples	spans	thr	precision	recall	f0.5	f1
prwp	7758	12359	0.70	0.8897	0.7777	0.8648	0.8299
fcv	7574	9693	0.70	0.8744	0.8542	0.8703	0.8642