Modelpublic

sign/Ettin-1B-WSD

sourceHugging Faceapache-2.0updated 19d agoView on Hugging Face

0likes40downloads

Model Card

Ettin-1B-WSD

Word sense disambiguation as multiple choice over WordNet (omw-en:1.4) definitions with a masked-LM answer slot, in a 1B-parameter encoder. Used by https://github.com/sign/word-sense-disambiguation (the WSDModernBertForMaskedLM class there adds a 128-way answer-letter decoder; answer_letters.json lists the letters). Same compact prompt template as sign/Ettin-150m-WSD and sign/ModernBERT-Large-Instruct-WSD.

Training (2026-09-07, "z1")

Fine-tuned from jhu-clsp/ettin-encoder-1b (ModernBERT architecture, 28 layers, hidden 1792) on generated sentences for 97k synsets, WordNet's own example sentences (minus a 5,000-example held-out slice), SemCor (222k instances, detokenized) and the Princeton WordNet Gloss Corpus (manual tags). Cosine schedule, label smoothing 0.1, weight decay 0.01, lr 2e-5, 2 epochs, batch 32 with 2 accumulation steps, fp32 weights with bf16 autocast. Weights are stored in bf16.

Benchmark	ModernBERT-Large-Instruct-WSD (c3, 395M)	this model (1B)
WordNet held-out slice (5,000 examples, seed 42)	78.3%	80.3%
SemEval "ALL" (Senseval-2/3, SemEval-07/13/15; 7,247 instances, any gold key)	80.6%	81.4%
Relative forward-pass cost	1.0	3.1

The most accurate model of the family; about three times the compute of the large model. It is also the distillation teacher of sign/Ettin-150m-WSD. As the second stage of a confidence cascade after the 150m (escalating the 9% least-confident prompts) it reaches 81.0% at 0.72x the cost of the large model.