temsa/OpenMed-PPSN-v4

temsa/OpenMed-PPSN-v4

Model Summary

This model extends OpenMed PII detection with Irish PPSN support.

• —Base: OpenMed/OpenMed-PII-SuperClinical-Large-434M-v1
• —Variant: classifier-only tuning with frozen original classifier rows
• —Output labels include all original OpenMed labels plus B-PPSN / I-PPSN

Intended Use

• —Clinical/admin text de-identification pipelines.
• —Hybrid use with checksum validator for PPSN is recommended.

Out-of-Scope

• —Not a standalone compliance/legal determination system.
• —Not validated on all healthcare institutions, note styles, or OCR artifacts.

Training Data

• —Synthetic PPSN dataset generated locally (scripts/build_dataset.py).
• —Additional benchmark source: nvidia/Nemotron-PII (license: CC-BY-4.0).

Evaluation

PPSN-focused test (synthetic holdout)

• —See scripts/eval_model.py output and dataset in data/ppsns_synth/test.jsonl.

Non-PPSN regression check (mixed set)

From reports/ab_non_ppsn_v4.json:

• —Base vs candidate agreement on real-text proxy set: F1 = 0.9814
• —Synthetic non-PPSN vs gold:
• —Base F1: 0.8701
• —Candidate v4 F1: 0.8420

Usage

Hybrid masking wrapper:

python3 scripts/mask_openmed_ppsn.py \
  --model models/openmed-ppsns-merged-v4 \
  --text "Patient John Doe, PPSN 1234567T, phone +353 87 123 4567." \
  --json

Limitations

• —Proxy real-text benchmark uses general text (wikitext) for drift checks.
• —Domain-specific local validation is required before production deployment.

Licensing and Attribution

• —Base model tag indicates Apache-2.0.
• —Dataset attribution required for CC-BY-4.0 sources.
• —Keep NOTICE and attribution links in derivative releases.

Reproducibility

• —Training script: scripts/train_lora.py
• —A/B script: scripts/ab_eval_non_ppsn.py
• —Environment lock: uv.lock

Release Metrics Snapshot

• —Synthetic non-PPSN F1 (base): 0.8701371396585503
• —Synthetic non-PPSN F1 (candidate): 0.8419874037788664
• —Real-set agreement F1 (candidate vs base): 0.9814140639969343
• —Entities/1k chars (base): 11.412409862256574
• —Entities/1k chars (candidate): 11.123105488147157

<!-- portfolio-comparison:start -->

Portfolio Comparison

Updated: 2026-03-16.

Use this section for the fastest public comparison across the temsa PII masking portfolio.

• —The first core table only includes public checkpoints that ship both comparable q8 accuracy and q8 CPU throughput.
• —The first PPSN table only includes public artifacts that ship comparable PPSN accuracy and CPU throughput.
• —Missing cells in the archive tables mean the older release did not ship that metric in its public bundle.
• —DiffMask rows use the reconciled clean_single_pass harness that matches the deployed runtime.
• —GlobalPointer rows use the public raw-only span-matrix release bundle and its packaged q8 ONNX artifact.
• —The same content is shipped as PORTFOLIO_COMPARISON.md inside each public model repo.

Irish Core PII: Comparable Public Checkpoints

Irish Core PII: Other Public Checkpoints

Finance-boundary q8 F1 is 1.0000 for OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc6, OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc7, OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc8, and all public IrishCore-DiffMask releases from rc1 to rc6. OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc5 ships 0.8750 on that public q8 suite.

PPSN-Only: Comparable Public Artifacts

PPSN-Only: Historical Public Checkpoints

If you need the strongest current raw-only Irish core model, start with IrishCore-GlobalPointer-135M-v1-rc4. If you need the fastest CPU-first raw-only line, compare it against IrishCore-DiffMask-135M-v1-rc6. If you need a PPSN-only artifact, compare the canonical fp32, fp16, and q8 variants of OpenMed-mLiteClinical-IrishPPSN-135M-v1 directly in the table above. <!-- portfolio-comparison:end -->