Modelpublic

kofdai/vera-distilgpt2-join

sourceHugging Facemitupdated 2mo agoView on Hugging Face

0likes18downloads

Model Card

Vera DistilGPT2 Join Partner (shared stereo-cross P)

Second model of the GPT-2 join suite: DistilGPT2 (6 layers, 768-d) distilled onto the same frozen shared basis `P` as `kofdai/vera-gpt2-matryoshka`.

Honest framing: prior-art model-stitching probe at personal scale — not a universal coordinate dictionary. P is reused from model A (reuse bias). Join ≫ random/shuffle is a weak lineage / shared-rep signal (python -m vera audit). Scale-up unverified (python -m vera about).

Used for cross-model coordinate join demos (Step 5 JOIN_VIABLE at GPT-2 class): mid-layer coords from GPT-2 → DistilGPT2 stay nearer solo than random/shuffle controls.

Bundle layout

File	Role
`model.safetensors`	Fine-tuned DistilGPT2 weights
`vera_basis.npz`	DistilGPT2 per-layer `means` + shared `P` (same dictionary as GPT-2 Matryoshka)
`config.json`	`kind=hook_container`, `role=join_partner`

Download with the GPT-2 suite

bash

python -m vera download-gpt2-suite
# pulls:
#   kofdai/vera-gpt2-matryoshka
#   kofdai/vera-distilgpt2-join

Or select GPT-2 (Vera suite) in python -m vera ui → Convert/Chat — both repos are fetched.

Proven join snapshot

Direction	Site	Real join vs solo	Random control
GPT-2 → DistilGPT2	late	~1.01× solo ppl	tens–hundreds× worse
DistilGPT2 → GPT-2	early	~1.07×	collapse

Details: CROSS_MODEL_JOIN.md in the Vera repo.

Load

python

from vera.runtime import load_from_hf
partner = load_from_hf("kofdai/vera-distilgpt2-join", rank=256)

Must be paired with the Matryoshka GPT-2 bundle that shares the same P columns.

Status

Research partner checkpoint for stereo-cross join — not a standalone chat model.