Reza2kn/netjets-g2p-benchmark-artifacts

NetJets G2P continuation bundle

This repository is the reproducibility bundle for the NameCoach NetJets pronunciation-distribution experiments.

Model

The matching adapter is in Reza2kn/t5gemma-2-4b-netjets-balanced-0p1, based on google/t5gemma-2-4b-4b. The uploaded checkpoint-2472 contains the final LoRA adapter, optimizer state, scheduler state, RNG state, tokenizer, and trainer state so training can resume.

This run used QLoRA (4-bit NF4 base weights), 4 GPUs, per-GPU batch size 8, gradient accumulation 1, learning rate 1e-5, and 0.1 epoch from the grouped-mixture adapter.

Data and evaluation

• —source/: original NetJets, sim-effective, and sim-final XLSX inputs.
• —data/: occurrence-level processed training data and source mixture metadata.
• —data_balanced_mixture_50_50/: occurrence rows plus grouped-distribution rows used in the balanced signal run.
• —eval/: sealed 1K evaluation inputs, direct-distribution outputs, sampling outputs, and score receipts.
• —rag/: SQLite pronunciation index and its manifest.
• —scripts/: builders, trainers, evaluators, and the RAG inference router.

The sealed 1K has no exact train-name overlap. Exact RAG lookup is authoritative only for known names; fuzzy retrieval is candidate-only for unseen names.

Important current results

The final adapter's 16-draw sealed sampling result is: parse 100%, top-1 target mode 50.1%, top-1 observed 58.1%, exact phonetic+IPA set 18.8%, exact variant count 34.2%, variant-set recall 55.6%, TV 0.5300.

The direct complete-distribution result is: parse 99.2%, exact full set 11.1%, exact variant count 36.3%, variant-set recall 50.1%, IPA mapping consistency 97.7%, TV 0.5653.

Resume outline

1. 1.Download the model repo and this bundle.
2. 2.Use scripts/build_balanced_distribution_mixture.py or the checked-in mixture files.
3. 3.Continue with scripts/train_t5gemma_distribution_multigpu.py and the uploaded adapter/checkpoint.
4. 4.Run both evaluators on the sealed inputs before comparing another run.
5. 5.Use scripts/infer_t5gemma_rag.py for exact-known-name routing and fuzzy candidate hints.

Do not use exact RAG on the sealed unseen-name score; report known-name and unseen-name routes separately.