mjbommar/opengloss-v2.0-lexicon
Superseded by OpenGloss v2.1 (2026-09-07): 109,633 lexemes and 250,003 live senses — twice this release's coverage — plus a new opengloss-v2.1-inflections form→lemma lookup. v2.0 stays published for reproducibility. OpenGloss v2.0 — Lexicon The entry-level view of OpenGloss v2.0: one row per lexeme, with everything that belongs to the entry rather than to one of its meanings — the kind discriminator, per-POS morphology, structured etymology, the lexical explanation, the… See the full description on the dataset page: https://huggingface.co/datasets/mjbommar/opengloss-v2.0-lexicon.
Superseded by [OpenGloss v2.1](https://huggingface.co/datasets/mjbommar/opengloss-v2.1-lexicon) (2026-09-07): 109,633 lexemes and 250,003 live senses — twice this release's coverage — plus a new opengloss-v2.1-inflections form→lemma lookup. v2.0 stays published for reproducibility.OpenGloss v2.0 — Lexicon
The entry-level view of OpenGloss v2.0: one row per lexeme, with everything that belongs to the entry rather than to one of its meanings — the kind discriminator, per-POS morphology, structured etymology, the lexical explanation, the encyclopedia article at every reading level it was written for, and the "X vs Y" contrast paragraphs written about the entry's symmetric relation edges. Sense-level content lives in opengloss-v2.0-senses, joined on lexeme_id.
Part of the OpenGloss v2.0 release family — 15 datasets built from one store of 54,724 lexemes and 137,314 live senses, all joinable on derived ids. See Related datasets for the rest.
What's new in v2.0 vs v1.3
- Schema v3. Every lexeme carries a
kinddiscriminator (simplex, compound, phrasal verb, idiom, proper noun, abbreviation, affix, function word); every sense carries a controlled domain leaf from a fixed ~160-leaf taxonomy instead of free text; every example carries the character span of the headword occurrence inside it. - Renditions, not one string. A definition is a set: the canonical one plus rewrites at four reading levels and in four registers, each produced in a single call from the canonical text so they say the same thing at different altitudes.
- A sense graph, not a word graph. Typed relations resolve to sense ids wherever the target's entry exists in the release, so
bank --hypernym--> financial institutionpoints at a meaning rather than at a string. - Retrieval data is first-class. Synthetic per-sense queries in eight styles, grounded QA pairs, mined word-in-context pairs, MS MARCO-style triples with graph-derived hard negatives, and graded TREC qrels — all derivable from, and consistent with, the same entries.
- Derivable identifiers everywhere. v1.3 published a positional id for lexemes and senses (
3d_model_noun_0) and nothing below that. v2.0 gives every rendition, edge, query, QA pair and provenance record an id computable from the row alone, and never renumbers: a retired sense is tombstoned, so the ids after it keep their meaning. - Per-field provenance. Which model wrote a field, how many tokens it took, what it cost — published as its own dataset.
Scope: fewer headwords, far more per headword
v2.0 is not a superset of v1.3. It covers 54,724 lexemes — a frequency-ranked subset of v1.3's 205,983 — and spends the difference on depth. If you need breadth of vocabulary, use v1.3; if you need graded renditions, resolved relations, spans, or retrieval supervision, use v2.0.
Key statistics
By tier
Coverage by tier
The release was built in three frequency-ranked passes and they did not all receive the same stages. This table is per-field and per-tier so the gaps are visible rather than averaged away.
Parts of speech
Domain roots
Files
Fields
54,724 rows, one row per lexeme.
One real row:
{
"lexeme_id": "aaa",
"headword": "aaa",
"language": "en",
"kind": "proper_noun",
"status": "complete",
"tier": "core",
"pos_list": [
"noun",
"interjection"
],
"sense_ids": [
"aaa:noun:0",
"aaa:noun:1",
"aaa:noun:2",
"… 3 more of 6"
],
"n_live_senses": 6,
"n_retired_senses": 0,
"is_stopword": false,
"frequency": 27053.0,
"zipf": null,
"morphology": [
{
"pos": "noun",
"plural": "aaas",
"past_tense": null,
"past_participle": null,
"present_participle": null,
"third_person_singular": null,
"comparative": null,
"superlative": null,
"derivations": [],
"collocations": [
"AAA rating",
"AAA battery",
"AAA membership",
"… 3 more of 6"
]
},
{
"pos": "interjection",
"plural": null,
"past_tense": null,
"past_participle": null,
"present_participle": null,
"third_person_singular": null,
"comparative": null,
"superlative": null,
"derivations": [],
"collocations": [
"loud exclamation",
"short exclamation",
"emotive utterance",
"… 2 more of 5"
]
}
],
"etymology": {
"summary": "The string aaa has no established lexical entry with a fixed historical etymology. The following structured trail treats aaa as (a) a potential nonce exclamation formed by vowel reduplication of a basic exclamation 'a' in English and (b) a separate path as a widely used initialism/acronym when uppercase. The resulting … [truncated for this card]",
"segments": [
{
"language": "English",
"language_code": null,
"form": "a",
"meaning": "interjection; basic exclamation or particle used to express emotion",
"era": "Old English–Present"
},
{
"language": "English",
"language_code": null,
"form": "aa",
"meaning": "lengthened or elongated vowel representation; onomatopoeic depiction of continuing exclamation",
"era": "Late 19th–20th century; common in written dialogue and cartooning; ongoing into digital text"
},
{
"language": "English",
"language_code": null,
"form": "aaa",
"meaning": "nonce string representing intensified vowel sequence; non-lexical in itself",
"era": "21st century–present"
},
"… 1 more of 4"
],
"cognates": [],
"references": [
"https://en.wikipedia.org/wiki/Reduplication",
"https://en.wikipedia.org/wiki/Interjection",
"https://www.aaa.com/about-us/history",
"… 2 more of 5"
]
},
"lexical_explanation": [
{
"reading_level": "neutral",
"register": "plain",
"text": "AAA is a noun used chiefly as an abbreviation for the American Automobile Association, a nonprofit federation of motor clubs in the United States and Canada that offers roadside assistance, travel services, insurance, and related consumer support. In financial and rating contexts, AAA also denotes the highest possible … [truncated for this card]",
"readability_grade": null
}
],
"encyclopedia": [
{
"reading_level": "neutral",
"register": "plain",
"text": "The string AAA operates as a cross-domain acronym and symbol in contemporary discourse, illustrating how a simple three-letter sequence can encode multiple high-salience concepts. While the all-lowercase form aaa often appears as a neutral placeholder or in instructional examples, the capitalized variant AAA signals do … [truncated for this card]",
"readability_grade": null
},
{
"reading_level": "grade_1",
"register": "plain",
"text": "AAA is a short name with many uses. The small form, aaa, can fill a blank spot. It can also appear in school work. The big form, AAA, has special uses in money, computers, shows, power cells, and sports. In money, AAA means a very safe pay score. It means a group can likely pay its bills. It does not mean no trouble wi … [truncated for this card]",
"readability_grade": 3.24
},
{
"reading_level": "grade_5",
"register": "plain",
"text": "AAA is a short name with many uses. The small form, aaa, can fill a blank or help in a lesson. The big form, AAA, has special uses in money, computer safety, shows, tools, and sports. In money, AAA means the best credit rank. It shows that a person or group can likely pay back what it owes. It cannot stop all money ris … [truncated for this card]",
"readability_grade": 4.59
},
"… 2 more of 5"
],
"contrasts": [
{
"edge_id": "aaa:interjection:0-synonym->ouch",
"target_sense_id": "ouch:interjection:0",
"verdict": "related_as_typed",
"text": "The interjections aaa and ouch can both be reflexive reactions to an unpleasant event, but they do not point to the same kind of discomfort. Ouch is the conventional choice when a minor physical injury or sensory insult hurts: “Ouch, I burned my finger.” It makes the source of the pain readily understood. Aaa is broade … [truncated for this card]"
},
{
"edge_id": "aaa:interjection:1-synonym->wow",
"target_sense_id": "wow:interjection:0",
"verdict": "related_as_typed",
"text": "The interjections aaa and wow both react to something unexpected, but they normally convey different attitudes toward it. Aaa is an alarmed or startled cry, suitable when a door slams or an event causes sudden concern. Wow usually marks surprise mixed with admiration, approval, or impressed interest, as when someone so … [truncated for this card]"
},
{
"edge_id": "aaa:interjection:1-synonym->gosh",
"target_sense_id": "gosh:interjection:0",
"verdict": "related_as_typed",
"text": "The interjections aaa and gosh can both respond to something unexpected, but they differ in force and typical use. Aaa is a sudden, attention-grabbing cry that can express alarm or an unprocessed shock, as when a door slams without warning. Gosh is a mild, controlled expression of surprise or emphasis, often used in co … [truncated for this card]"
}
],
"provenance_summary": {
"models": [
"gpt-5.4-nano",
"gpt-5.6-luna",
"rule:classify_kind_deterministic",
"… 4 more of 7"
],
"total_cost_usd": 0.01544655,
"n_records": 79
},
"created_at": "2025-11-28T11:11:31.136276+00:00",
"updated_at": "2026-09-04T10:47:25.023876+00:00"
}Loading it
from datasets import load_dataset
ds = load_dataset("mjbommar/opengloss-v2.0-lexicon", split="train")
print(ds)
print(ds[0])The shards are plain parquet, so nothing forces you through datasets — read them straight, locally or over hf://:
import polars as pl
df = pl.read_parquet("hf://datasets/mjbommar/opengloss-v2.0-lexicon/data/train-*.parquet")
print(df.head())import duckdb
duckdb.sql("SELECT count(*) FROM 'hf://datasets/mjbommar/opengloss-v2.0-lexicon/data/train-*.parquet'").show()Entries whose encyclopedia article was written for grade 5
import polars as pl
lex = pl.read_parquet("data/train-*.parquet")
grade5 = (
lex.explode("encyclopedia")
.filter(pl.col("encyclopedia").struct.field("reading_level") == "grade_5")
.select("headword", pl.col("encyclopedia").struct.field("text"))
)
print(grade5.head())Identifiers, and how they compose
Every id is derived from structure, never randomly minted, so a consumer can recompute one from a row and join across the whole family without a lookup table. Sense positions are stable across regenerations: a retired sense is tombstoned, not removed, so the indices after it never shift.
An edge id keys on the target's slug, not on the target's sense, so resolving a target never changes the id of the edge that found it.
Reading levels and registers
A rendition is keyed on a (reading_level, register) pair. The canonical rendition of every field is (neutral, plain); everything else is a rewrite of it.
marketing sits on the register axis for convenience but is a genre value rather than a point on the formality scale — worth remembering if you train a formality classifier on this column.
Related datasets
Everything below is built from the same store and joins on lexeme_id / sense_id.
Known limitations
- It is synthetic. Every string here was written by a language model against a schema, not transcribed from a corpus or checked by a lexicographer. It is well-formed and internally consistent; it is not attested usage, and it will contain confident errors. Do not use it as ground truth about what a word means.
- Judge scores 70.2/100 (core + tier 2) and 66.7/100 (tier 3). A different model family (Claude Opus) scored fixed 40-entry stratified samples at the close of each build. Sample statistics, not per-entry guarantees, and the judge is itself a model.
- Relation precision is the weakest axis. Relations were judged for validity and the ones that failed were demoted rather than asserted; symmetric reciprocity finished at 98.0% for synonyms and 99.1% for antonyms, and 3,709 senses were left with no relation at all. Treat a single edge as a hypothesis, not a fact; treat the aggregate graph as usable.
- Tier 3 is deliberately partial. 12,838 lexemes received the text stages (glosses, examples, encyclopedia) but not the queries, QA pairs, contrasts or register renditions. The coverage table above gives the exact per-field share; nothing is hidden behind an average.
- The encyclopedia is entry-level. One article per headword, about the headword as a whole. On a polysemous entry it is not a description of any one sense, and it is never used as a positive for one (D-71). It is entry-level reference prose, not a specialist article.
Citation
@misc{bommarito2025opengloss,
title = {OpenGloss: A Synthetic Encyclopedic Dictionary and Semantic Knowledge Graph},
author = {Bommarito, Michael J., II},
year = {2025},
eprint = {2511.18622},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2511.18622}
}License
Released under Creative Commons Attribution 4.0 International (CC-BY 4.0). Attribution to the OpenGloss project is required; commercial use is permitted.