datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
circuitlens-gemma-2-2btranscoder-descriptions-and-evaluations
CircuitLens & WeightLens: Transcoder Descriptions and Evaluations
This dataset contains automatically generated descriptions and evaluation metrics for Gemma-2-2B transcoders, produced using CircuitLens and WeightLens methods.
Methods
CircuitLens: https://github.com/egolimblevskaia/CircuitLens
WeightLens: https://github.com/egolimblevskaia/WeightLens
Dataset Structure
The dataset is organized by layers (0, 4, 7, 10, 12, 15, 18, 21, 23, 25), with each layer… See the full description on the dataset page: https://huggingface.co/datasets/egolimblevskaia/circuitlens-gemma-2-2btranscoder-descriptions-and-evaluations.diffing-stats-gemma-2-2b-crosscoder-l13-mu4.1e-02-lr1e-04 Contains maximum activating examples for all the features of our crosscoder trained on gemma 2 2B layer 13 available here: https://huggingface.co/Butanium/gemma-2-2b-crosscoder-l13-mu4.1e-02-lr1e-04/blob/main/README.md
base_examples.pt contains all the maximum examples of the feature on a subset of validation test of fineweb
chat_examples.pt is the same but for lmsys chat data
chat_base_examples.pt is a merge of the two above files.
All files are of the type dict[int, list[tuple[float… See the full description on the dataset page: https://huggingface.co/datasets/science-of-finetuning/diffing-stats-gemma-2-2b-crosscoder-l13-mu4.1e-02-lr1e-04.max-activating-examples-gemma-2-2b-l13-ckissanediffing-stats-SAE-difference_cb-gemma-2-2b-L13-k100-x8-lr1e-04-local-shufflingdiffing-stats-SAE-base-gemma-2-2b-L13-k100-x32-lr1e-04-local-shufflingdiffing-stats-SAE-difference_cb-gemma-2-2b-L13-k100-lr1e-04-local-shufflingdiffing-stats-SAE-difference_cb-gemma-2-2b-L13-k100-x2-lr1e-04-local-shufflingdiffing-stats-SAE-chat-gemma-2-2b-L13-k100-lr1e-04-local-shufflingdiffing-stats-SAE-difference-gemma-2-2b-L13-k100-lr1e-04-local-shufflingdiffing-stats-gemma-2-2b-L13-k100-lr1e-04-local-shuffling-CCLossdiffing-stats-SAE-difference_cb-gemma-2-2b-L13-k100-x1-lr1e-04-local-shufflingdiffing-stats-gemma-2-2b-L13-k100-lr1e-04-local-shuffling-Decoupledgemma-2-2b-it-ipg-training-datadiffing-stats-gemma-2-2b-it-Meditron3-L16-k100-lr1e-04-local-shuffling-CCLossdiffing-stats-gemma-2-2b-it-Meditron3-L16-mu3.8e-02-lr1e-04-local-shuffling-CCLossdiffing-stats-gemma-2-2b-gemma-2-2b-it-L13-mu5.5e-02-lr1e-04-local-shuffling-CrosscoderLossdiffing-stats-gemma-2-2b-gemma-2-2b-it-L13-k256-lr1e-04-local-shuffling-Crosscoderdiffing-stats-gemma-2-2b-L13-mu5.2e-02-lr1e-04-2x100M-local-shuffling-SAELossdiffing-stats-gemma-2-2b-L13-k100-lr1e-04-local-shuffling-SAELossdiffing-stats-gemma-2-2b-gemma-2-2b-it-L13-mu2.5e-02-lr1e-04-local-shuffling-CrosscoderLossdiffing-stats-gemma-2-2b-L13-mu5.2e-02-lr1e-04-2x100M-local-shuffling-SAELoss_model_final.ptdiffing-stats-gemma-2-2b-gemma-2-2b-it-L13-k55-lr1e-04-local-shuffling-Crosscoder
