Modelpublic

Prisma-Multimodal/DINO-vanilla-x64-all_patches_11-resid_post-1085-99

sourceHugging Faceupdated 1y agoView on Hugging Face

0likes15downloads

Model Card

DINO Sparse Autoencoder Checkpoint

This model is a sparse autoencoder trained on DINO's internal representations.

Model Details

Architecture

—Layer: 11
—Layer Type: hookresidpost
—Model: facebook/dino-vitb16
—Dictionary Size: 49152
—Input Dimension: 768
—Expansion Factor: 64
—CLS Token Only: False

Training

—Learning Rate: 0.000299301838857893
—Batch Size: 4096
—Context Size: 197

Sparsity

—L0 (Active Features): 1085

Additional Information

—Wandb Run: https://wandb.ai/perceptual-alignment/DINO-SAE-Vanilla-Sweep/runs/ef3ljdzk
—Random Seed: 42