CoolFace
Datasetpublic

alliedtoasters/latenet-v0-activations-llama3.1-405b-base

meta-llama/Llama-3.1-405B — Activation Dataset Cached activations extracted from meta-llama/Llama-3.1-405B (revision b906e4dc842aa489c962f9db26554dcfdde901fe). LateNet v0 activations for Llama 3.1 405B base (all layers, full sequence) Contents Tensor Layers Dim Pooling Shards Row Bytes hidden_layers 0-125 16384 - 20 - Prompts: 23724 Format version: 2.0 Load with lmprobe from lmprobe import load_activations, Probe acts =… See the full description on the dataset page: https://huggingface.co/datasets/alliedtoasters/latenet-v0-activations-llama3.1-405b-base.

sourceHugging Facecc-by-4.0updated 6mo agoView on Hugging Face
0likes919downloads
Dataset Card

meta-llama/Llama-3.1-405B — Activation Dataset

Cached activations extracted from `meta-llama/Llama-3.1-405B` (revision b906e4dc842aa489c962f9db26554dcfdde901fe).

LateNet v0 activations for Llama 3.1 405B base (all layers, full sequence)

Contents

TensorLayersDimPoolingShardsRow Bytes
hidden_layers0-12516384-20-
  • —Prompts: 23724
  • —Format version: 2.0

Load with lmprobe

python
from lmprobe import load_activations, Probe

acts = load_activations("alliedtoasters/latenet-v0-activations-llama3.1-405b-base", layers=[0])
probe = Probe(classifier="logistic_regression", random_state=42)
probe.fit_from_activations(acts[0], labels)

Load without lmprobe (standalone)

python
import json
import pyarrow.parquet as pq
from safetensors import safe_open

# Load the index — all metadata is embedded in the Parquet schema
table = pq.read_table("index/train-00000-of-00001.parquet")
df = table.to_pandas()
meta = json.loads(table.schema.metadata[b"lmprobe:tensors"])

# Get layer 0 activation for prompt 0
row = df.iloc[0]
pattern = meta["hidden_layers"]["file_pattern"]
path = pattern.format(layer=0, shard=row["shard_index"])
with safe_open(path, framework="pt") as f:
    vec = f.get_tensor("hidden.layer_0")[row["row_offset"]]
    # vec.shape: (16384,)
Full-sequence dataset: The shard_index / row_offset columns always address the last-token pooled vector. For per-token access, use the token_shard_ids and token_shard_offsets list columns — see the lmprobe:tensors schema metadata for details.

Load with HF Datasets

python
from datasets import load_dataset

# Shows prompt text + labels in Dataset Viewer
ds = load_dataset("alliedtoasters/latenet-v0-activations-llama3.1-405b-base")
print(ds["train"][0])  # {"text": "...", "label": ..., ...}

Provenance

  • —lmprobe version: 0.9.2
  • —Extraction backend: local
  • —Created: 2026-04-04T16:35:14.904340+00:00
  • —PyTorch: 2.11.0+cu130
  • —Transformers: 5.4.0