Modelpublic

arturoornelasb/triadic-gpt2-medium-v6

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face

0likes

Model Card

Triadic GPT-2 Medium (V6 — Extended Corpus)

A 72-bit triadic projection head trained on top of frozen GPT-2 Medium (345M params), scaled from 72 core primitives to 2,166 concepts spanning 14 scientific domains. Same architecture as v5_frozen but with a much denser anchor field.

![Paper DOI](https://doi.org/10.5281/zenodo.19375167) ![Repo DOI](https://doi.org/10.5281/zenodo.19374914)

What is this?

This model adds a triadic head to GPT-2 Medium that projects language model embeddings into 72-bit binary codes representing ontological primitives from the Emergent Duality framework.

V6 extends the gold file from 72 primitives to 2,166 concepts (72 primitives + 2,094 derived from a first-principles corpus across 14 sciences). The denser field forces 48 bits to be active (vs. 4 in v5), yielding more discriminative codes at the cost of unique-signature collapse (many concepts share signatures because the 72-bit space is saturated).

Files

File	Description
`best.pt`	Final trained weights (best checkpoint at 100K steps)
`gold_extended_v6.json`	Gold standard: 2,166 concept targets (72 primitives + 2,094 derived)
`run_config.json`	Full training configuration
`results.json`	Final metrics
`training_log.csv`	Training curve (loss, accuracy per step)

Key Metrics

Metric	Value
Bit accuracy (test)	80.9%
Subsumption accuracy	95.2%
PPL (general text)	31.95 (still identical to baseline after 100K steps)
Active bits	48/72 (vs. 4 in v5 — field density forces utilization)
Dead bits	37/72
Entropy (bits)	0.370 (9.5× v5's 0.039)
Unique signatures	21.4% (463 unique out of 2,166)
Regla de tres (cosine)	0.998 (8 quadruples)
Discovered dependencies	501
Triadic dependencies	1,714
Trainable params	73,728 (0.02% of total)
Training time	12.3h on RTX 4060 Ti 16GB

Phase transition

Phase transition occurs at step 55,000 (warmup boundary). Layer emergence is inverted: L6 emerges before L1 in v6, contrary to the theoretical L1 → L6 ordering. Combined with v8/v9 X-ray analysis, this supports the cascade-not-ordering reformulation of Q1.

Architecture

—Base model: GPT-2 Medium (gpt2-medium, 355M params, frozen)
—Triadic head: Linear(1024 → 72) with iFSQ activation ($2\sigma(1.6x) - 1$)
—Loss: $\mathcal{L}\text{lang} + 0.05 \cdot (\mathcal{L}\text{tri} + 2\mathcal{L}\text{sup} + 5\mathcal{L}\text{sub})$

Usage

python

from transformers import GPT2Model, GPT2Tokenizer
import torch

tokenizer = GPT2Tokenizer.from_pretrained("gpt2-medium")
gpt2 = GPT2Model.from_pretrained("gpt2-medium")
head_state = torch.load("best.pt", map_location="cpu")
# See repo code for the TriadicHead module definition.

Full inference pipeline: see dualidademergente+reptimeline/model/triadic.py in the GitHub repo.

Training

bash

python train.py --bits 72 --gold-file gold_extended_v6.json \
  --run-name gpt2_triadic_72_v6 --freeze-base --steps 100000

Companion models

—`triadic-gpt2-medium-v5`: Same architecture, 72 core primitives only.
—`triadic-gpt2-medium-v8`: Deep head, 250K steps, base for V8 telescopic X-ray.
—`triadic-gptneo-125m-v9`: GPT-Neo 125M / OpenWebText (cross-architecture replication).

Citation

bibtex

@software{ornelasbrand2026triadic,
  author = {Ornelas Brand, J. Arturo},
  title = {Triadic Emergent Duality: 14+ Candidate Dualities Across 6 Algebraic Layers},
  year = {2026},
  doi = {10.5281/zenodo.19374914},
  url = {https://github.com/arturoornelasb/triadic-emergent-duality}
}

License

Apache 2.0 for these weights. The companion code repository is BUSL-1.1.