Prisma-Multimodal/DINO-topk-x64-all_patches_6-resid_post-768-98

DINO Sparse Autoencoder Checkpoint

This model is a sparse autoencoder trained on DINO's internal representations.

Model Details

Architecture

• —Layer: 7
• —Layer Type: hookresidpost
• —Model: facebook/dino-vitb16
• —Dictionary Size: 49152
• —Input Dimension: 768
• —Expansion Factor: 64
• —CLS Token Only: False

Training

• —Learning Rate: 0.00010660139494110328
• —L1 Coefficient: -
• —Batch Size: 4096
• —Context Size: 197

Checkpoint details

• —Checkpoint: 1/10

Sparsity

• —L0 (Active Features): 768

Additional Information

• —Wandb Run: https://wandb.ai/perceptual-alignment/dino-sae-sweep/runs/css0ei8r
• —Random Seed: 42