hoomeh/efficient-cot

To be written. <!-- # pretty_name: "Efficient CoT — patch-labeled Llama-3.1-8B CoT activations"

license: llama3.1

tags:

- interpretability

- mechanistic-interpretability

- activations

- chain-of-thought

- activation-patching

- llama

language:

- en

---

# Efficient CoT — patch-labeled chain-of-thought activations

Hidden states of Llama-3.1-8B-Instruct collected from greedy chain-of-thought (CoT)

generations on 26 reasoning tasks, where every stored state carries a causal label:

whether activation-patching that state into the same question's direct-answer (no-CoT)

run rescued the correct answer.

This is the training corpus for the activation-probe experiments of the Efficient CoT

project (thesis code release forthcoming): probes trained on these states predict

patch-rescue success, and the accompanying transfer/subspace analyses characterize where

and in what form the "successful reasoning" signal lives in the residual stream.

## Contents

| | |

|---|---|

| Model | meta-llama/Llama-3.1-8B-Instruct (hidden size 4096) |

| Tasks | 24 BIG-Bench-Hard tasks + GSM8K + StrategyQA (26 directories) |

| Coverage | BBH: up to 250 test questions per task; GSM8K: all 1,319 test questions; StrategyQA: all 2,290 |

| Layers | 0, 2, 4, …, 30, 31 (17 layers) |

| Files | 9,370 HDF5 files, ~176 GiB (float16, gzip) |

One file per question: llama8B/<dataset>/sample_<idx>.h5.

Note: bbh_salient_translation_error_detection is included for completeness but was

excluded from all probe training and reported results (its traces are degenerate — most

collapse to 5-token stubs). Experiments use the remaining 25 tasks.

## File format

Root attributes carry full provenance: cot_text (the complete CoT generation),

dataset, sample_idx, gold_answer, source_generated_answer, model_name,

seed, num_tokens, max_gen_len, and prompt/token-id hashes.

Each layer group layers/<L> holds three row-aligned datasets (N = stored token

positions of the question's CoT trace):

| dataset | shape | dtype | meaning |

|---|---|---|---|

| hidden_states | (N, 4096) | float16 | residual-stream state at layer L, position p |

| source_positions | (N,) | int32 | trace token index p of each row |

| labels | (N,) | bool | patch outcome (below) |

Label semantics. For each stored source position, the layer-L hidden state from the

CoT trace was patched into the same question's direct-answer run (no CoT) at a fixed

target position, and the run was regenerated. labels[i] = True iff the patched run

produced the correct answer — a rescuing state. Labels are joined into these files at

collection time from the underlying per-patch results; probe training reads only these

files.

## Loading

```python

from huggingfacehub import snapshotdownload

import h5py

root = snapshot_download(

repo_id="hoomeh/efficient-cot",

repo_type="dataset",

allow_patterns=["llama8B/gsm8k/*"], # fetch one task, or omit for all

)

with h5py.File(f"{root}/llama8B/gsm8k/sample_0000.h5") as f:

print(dict(f.attrs))

g = f["layers/16"]

states, labels, positions = g["hiddenstates"][:], g["labels"][:], g["sourcepositions"][:]

```

The project code accepts this repo directly as a cache backend — set

hs_cache_dir=hf://hoomeh/efficient-cot and the needed task subfolders are downloaded

on demand (read-only, incremental).

Question-level train/val/test splits (75/5/20) are not stored here; they are derived

deterministically in the code from seed=42 and the frozen 26-task codebook, so all

published splits reproduce exactly from this repo plus the code.

## Provenance & licenses

Activations are derived from Llama-3.1-8B-Instruct (Llama 3.1 Community License).

Source questions come from BIG-Bench-Hard (MIT),

GSM8K (MIT), and

StrategyQA (MIT). CoT generations were produced

with greedy decoding (max_gen_len per file attributes, typically 400).

## Citation

Citation entry will be added with the code release. -->