rmems/Spikenaut-SNN-Telemetry

🧠 Spikenaut SNN Telemetry Dataset

"The threshold at which stimulus becomes perceptible"

Telemetry for the Spikenaut Supervisor control stack: v3 restructures this corpus from time-series forecasting into an action-proposal trajectory dataset — states, proposed actions, safety-filter verdicts, and outcomes — while every v2 config remains published, byte-identical and loadable.

The control hierarchy this dataset serves:

learned policy → action proposal → deterministic safety filter → execution → logged trajectory

The deterministic safety filter is always the final authority. Spikenaut (the learned SNN policy) only proposes. This dataset is structured so that training, evaluation, and audit all respect that boundary.

📊 Dataset Overview

v3 trajectory configs (Parquet, splits: train / validation / test)

gpu_telemetry_v3 and qubic_signals are flat corpus derivations — they deliberately cover all episodes (train, validation, test, and embargo), so they publish under a single full split rather than a misleading train label. Episode membership is row_index // 4096 against the block boundaries in Episode & Split Design below. For leakage-safe training, use state_telemetry.

Two further v3 tables ship as typed, empty Parquet schema artifacts and are deliberately not listed as loadable configs yet:

They are excluded from the viewer configs because datasets (≤ 5.0.x) cannot batch a 0-row Parquet file (ArrowInvalid: BatchSize must be greater than 0). Each gains a config entry with its first populated release; the schemas are already frozen in the Parquet files and in `v3_build.py`.

v2 configs (same records, still authoritative for the raw captures)

These are disjoint capture windows, not one continuous run.

Serving format: the v2 configs load from verified Parquet conversions under v2_parquet/ — a script-less Hub dataset gets exactly one builder, inferred from the first config's files, so JSONL and Parquet configs cannot share a card (see the 2026-08-16 changelog). The original JSONL files remain in full_data/, byte-untouched and canonical; the conversions are produced and fidelity-checked by the ETL (identical rows, columns, and nulls; the one representation change is qubic_ticks.timestamp: timestamp[s] → timestamp[ms], same values at finer unit).

from datasets import load_dataset

state = load_dataset("rmems/Spikenaut-SNN-Telemetry", "state_telemetry")   # train/validation/test
outcome = load_dataset("rmems/Spikenaut-SNN-Telemetry", "outcomes")
enc = load_dataset("rmems/Spikenaut-SNN-Telemetry", "encoding_params", split="train")
v2 = load_dataset("rmems/Spikenaut-SNN-Telemetry", "mining", split="train")

All v3 trajectory tables join on (`episode_id`, `step_idx`); ts_utc is the time key where it exists.

🛡️ Control Hierarchy & Safety

• —The deterministic layer is final. A learned proposal is data into the safety filter, never a command. safety_filter_log.filter_verdict records what the filter did with each proposal: allow, modify, veto, or substitute, plus the filter_rule_id that fired.
• —Deployment modes (safety_filter_log.mode): shadow (policy proposes, filter logs, humans/heuristics act), assist (filtered proposals surface as suggestions), active (filtered proposals execute). Promotion between modes is a human decision informed by this dataset.
• —Shadow KPI: filter-override rate — the share of proposals the filter did not pass through unchanged (verdict != "allow"). A policy is not a candidate for assist while its override rate is materially nonzero on the scenarios that matter.
• —override / override_reason record the rare, logged case of a human overriding the filter — the field exists so that such events are data, not folklore.

📐 Telemetry Contract

• —`schema_version` (column in every v3 config, currently 3.0.0) is the semantic version of the telemetry contract: PATCH = docs/units clarifications, MINOR = additive nullable columns, MAJOR = anything breaking (rename, retype, resample).
• —Missing means `null` — never 0.0, never "". Columns whose collectors do not exist yet are present, typed, and null (schema first, backfill later).
• —`synthetic` flags rows not produced by a real collector; the entire current backfill is measured data, so it is false everywhere. regime is null in the backfill: classifying idle/ramp/overload from values would be inference, not provenance.

Sampling rates and time

No timestamp in this dataset is ever synthesized. ts_utc is nullable int64 nanoseconds since epoch. ts_synthetic_offset_ms was designed as row_index × documented sampling interval; because no sampling interval is documented for the GPU capture, it ships null rather than encoding a guess. (The dataset's own history shows why: 114,250 fabricated timestamps were once published as a real collection window. Never again.) When the cadence is confirmed from the collector source, populating the column is a MINOR bump. Ordering within an episode is still exact via step_idx.

state_telemetry columns (52)

\* sm_clock_mhz is backfilled from v2 gpu_clock_mhz (the collector sampled a single graphics clock; NVML SM and graphics clocks are reported separately on paper but were one reading here). The redundant v2 clock_mhz duplicate was verified equal and dropped.

Q8.8 convention

FPGA-side quantities use Q8.8 signed fixed point: value = int16 / 256, range −128.0 … +127.99609375, 1 LSB = 1/256 ≈ 0.0039. fpga_membrane_q88 stores the raw int16; encoding_params.q8_8_scale = 256 records the scale so any backend can reproduce the mapping. This matches the Limen-Neural hardware contract (256-neuron / 1024-weight envelope, .mem export in Q8.8).

🎛️ Action Taxonomy

*_action / *_action_id columns use this frozen mapping (append-only, never renumbered):

Deterministic teacher rules (v1.0.0)

Implemented and unit-tested in `teacher_policy.py`; every label records the rule that produced it in action_proposals.teacher_rule_id, with label_source = "teacher_rule" and label_confidence = 1.0 (deterministic rules are certain by definition). First match wins:

Rules key off the NVML throttle bitmask, never hard-coded temperatures — a GPU at 85 °C with no slowdown bit set is a GPU doing its job. Idle (0x1) and sw_power_cap (0x4) bits describe normal operation and map to no_op. If `throttle_reasons` or `ecc_dbe_vol` is null, the teacher emits no label at all — which is why action_proposals is currently empty (see Migration).

⚡ Encoding

Policy: store raw values plus encoding parameters — never only pre-encoded spike trains. Spikes are a lossy function of encoder settings; raw + params lets `axon-encoder` (or any backend) re-encode for a different simulator, timestep, or hardware target.

encoding_params is the sidecar (one row per state feature):

• —encoder_type ∈ {rate, delta, latency, population, poisson, temporal, derivative} — the default assignment is rate for continuous magnitudes and delta (threshold = 1 count) for monotone counters.
• —min / max — the encoder input range, fitted on the train split only (leakage rule); null for features with no data yet.
• —base_rate_hz / max_rate_hz — firing rates mapped to range endpoints (defaults 5 → 100 Hz), matching RateEncoder::try_new(base_rate_hz, max_rate_hz, (min, max), dt_seconds).
• —dt_seconds — the encoder's replay integration step (default 0.010). This is a prescriptive encoding choice, not a claim about collection cadence (which is undocumented for the GPU source). Batch encoding uses p = 1 − exp(−rate_hz · dt_seconds).
• —q8_8_scale = 256 — see Q8.8 convention above.

🧩 Episode & Split Design

• —Episodes are fixed 4,096-step windows over the verified-contiguous row_index order (gpu-000000 … gpu-000198; the final episode is short: 2,965 steps). The capture has no wall clock, so bounded windows — not load-event segmentation — are the honest boundary choice for the backfill.
• —RLDS flags live in outcomes: is_first / is_last mark episode edges; episodes end by windowing, so is_last = true is truncation and is_terminal stays false; discount = 1.0.
• —Outcome horizon H = 64 steps. d_gpu_temp_c[t] = gpu_temp_c[t+64] − gpu_temp_c[t], computed strictly within an episode; the last 64 steps of each episode are null. The other deltas (d_ttft_p99_s, d_tokens_per_s, d_kv_cache_usage) and the event flags await their collectors and are null. reward is null: no reward function is defined yet, and publishing one implicitly through data would bypass review.
• —Splits are chronological blocks (≈70/15/15 by episode): train = gpu-000000…gpu-000138, validation = gpu-000140…gpu-000168, test = gpu-000170…gpu-000198. Episodes gpu-000139 and gpu-000169 (8,192 rows) are embargo gaps published in no split, so blocks are never temporally adjacent and no 64-step outcome window can cross a boundary.
• —Leakage rules: never random-shuffle across time; no episode spans two splits; fit normalization/encoding statistics (including encoding_params min/max) on train only; anything fitted elsewhere is a bug.

🔍 v2 Schemas

<details> <summary>Click to expand the v2 config documentation (unchanged from the v2 card)</summary>

gpu_telemetry — neuromorphic_data.jsonl

12 sensor columns + row_index. This source carries no timestamp — the collector never emitted one; records are ordered but not time-located, and a synthetic clock is deliberately not supplied. qubic_tick_trace, qubic_tick_rate, qubic_epoch_progress ride along in this file for continuity; v3 splits them into qubic_signals.

mining — node_sync_harvest.jsonl

Chain attribution comes from the source's timestamp field, which takes four forms:

114,238 rows carry no chain label — that information does not exist in the source. They are null, never "".

hft — ghost_market_log.jsonl

25 columns of paper-trading state. Actions: buy 14,603 / sell 14,580 / observe 2,390. Simulated trading — not live capital.

qubic_ticks — qubic_ticks_snn.jsonl

The `_derived` columns are not measurements. They are a fixed function of tick_rate, kept for continuity. The independent signals are tick_rate and qubic_tick_trace.

Model artifacts — ⚠️ provenance unverified

full_data/snn_model.json, full_data/hybrid_training_results.json, and models/mining_v2/*.mem are not reproducible from this repository and show signatures consistent with untrained placeholders (identical weight vectors, denormal weights, contradictory thresholds). They are under audit. Do not treat them as trained parameters; the previously published 95.2% accuracy figure has no reproducible basis.

</details>

📈 Provenance

rmems/Theseus-Quarry           Rust collectors → raw JSONL
        ↓
rmems/spikenaut-telemetry-etl  ingest → validate → clean → publish       (full_data/ JSONL)
        ↓                       └─ spikenaut-etl build-v3 ─────────────→ (v3/ + v2_parquet/)
rmems/Spikenaut-SNN-Telemetry  this dataset
        ↓
rmems/Spikenaut-SNN            model training

Every file here is generated. Do not hand-edit them; report data issues against the ETL repository. The v2 gates (no constant/all-null columns, distinct-row ratio, bounded drift, non-fabricated timestamps, exact schema match) still run; the v3 builder adds its own non-degeneracy asserts and refuses to write a broken tree.

🔀 Migration: v2 → v3

Everything v2 is preserved. Same files, same configs, same bytes; v3 is purely additive under v3/. If you consume gpu_telemetry, mining, hft, or qubic_ticks today, nothing changes for you.

📝 Changelog

2026-08-16 — all configs Parquet-served; viewer restored

The 2026-08-15 release broke the Dataset Viewer for every v3 config (JSON parse error: Invalid value. in row 0): a script-less Hub dataset resolves one packaged builder from the first config's data files (datasets/load.py, HubDatasetModuleFactoryWithoutScript) and applies it to every config — so the JSON builder was fed Parquet bytes. Mixed JSONL/Parquet configs cannot work on the Hub.

Since the v3 contract mandates Parquet, the four v2 configs now load from Parquet conversions under v2_parquet/, generated and fidelity-verified by spikenaut-etl build-v3 (ETL PR #9): identical row counts, rows, columns, and nulls, with one representation change — qubic_ticks.timestamp is timestamp[ms] instead of timestamp[s] (Parquet has no seconds resolution; values unchanged). full_data/*.jsonl are byte-untouched and remain the canonical cleaned exports. Same-day earlier fix: dataset_info dtypes were serialized as repr strings (Value('float64')), which the viewer's config-names step cannot parse; now generated with Features._to_yaml_list() and round-trip-validated before every card commit.

2026-08-15 — v3.0.0: action-proposal trajectory restructure

Built by `spikenaut-etl build-v3` (PR #7, merged as 983323b), teacher policy v1.0.0.

• —New configs: state_telemetry, outcomes, gpu_telemetry_v3, qubic_signals, encoding_params; schema artifacts action_proposals, safety_filter_log (typed, empty — see Overview). Task category reinforcement-learning added; time-series-forecasting retained for the v2 configs. The metrics: accuracy frontmatter key was removed — it referred to the disavowed 95.2% figure (see Model artifacts).
• —Zero teacher labels emitted over the v2 backfill — the historical GPU capture has no NVML throttle mask and no ECC counters, and the teacher does not guess. This is the intended, honest result; labels begin when collectors publish the core signals.
• —ts_utc / ts_synthetic_offset_ms ship null for the GPU-derived configs: no sampling interval is documented anywhere for that capture, and this dataset does not fabricate clocks (see 2026-08-03 entry for what happened last time someone did).
• —Step-0 verification notes, for the record: mining, hft, and qubic_ticks do carry real in-band timestamps (only gpu_telemetry is time-blind); hft has 25 columns; clock_mhz was verified byte-equal to gpu_clock_mhz across all 813,973 rows before being dropped from gpu_telemetry_v3; row_index was verified contiguous 0…813,972.
• —Decisions taken where the spec left room, recorded here: episodes are fixed 4,096-step windows (no wall clock to anchor load events); embargo = one full episode per boundary (≥ H = 64); regime left null in backfill rather than inferred; sm_clock_mhz backfilled from v2 gpu_clock_mhz; v2 mining / hft / qubic_ticks keep their in-band timestamps and are not duplicated into v3 configs.

2026-08-04 — 12 placeholder rows removed from mining

Trailing rows appended out of order (~21 h backwards), the only rows carrying a UTC offset, holding two distinct (power_w, gpu_temp_c) pairs. Quarantined; the pipeline now fails the build on any unquarantined time reversal. mining 120,334 → 120,322.

2026-08-03 — data rebuilt from recovered originals

neuromorphic_data.jsonl (813,973 rows of {"telemetry":{}}) and node_sync_harvest.jsonl (every numeric 0.0) were rebuilt from recovered originals. Cause: a Symbol/String key mismatch in the old Julia cleaning script; every filter matched nothing, every lookup returned its default, and nothing asserted the output was non-degenerate. The same script overwrote 114,250 real timestamps with a fabricated base + 10s × index sequence. Also: chain attribution recovered (dynex 5,001 / qubic 1,083); qubic derived columns renamed *_derived; samples regenerated as seeded random draws; 17 dead GPU columns and 13 dead mining columns dropped; model artifacts flagged unverified. If you pulled a revision before 2026-08-03, re-download.

📜 Citation

@dataset{spikenaut_snn_telemetry,
  author={Montoya Cardenas, Raul},
  title={Spikenaut SNN Telemetry Dataset},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/datasets/rmems/Spikenaut-SNN-Telemetry}
}

⚖️ License

MIT OR Apache-2.0 — see LICENSE. Dual-licensed: use whichever fits your project.

🙏 Acknowledgments

• —Kaspa, Monero, Qubic, Quai, Dynex, Verus communities for open-source node implementations
• —E-prop authors (Bellec et al., 2020) and STDP pioneers (Bi & Poo, 1998)
• —vLLM for the inference-server metric vocabulary mirrored in state_telemetry

Built by Raul Montoya Cardenas — WGU AI Engineering