ppak10/Agentic-SLS-Telemetry

Inova-Mk1-Telemetry

Time-aligned printer-state recordings from Inova Mk1 SLS 3D print runs. One row per 10 Hz tick — the recorder's /state/snapshot poll — with the full sensor state snapshot (~64 columns: temperatures, position, power, lights) on every row, the nearest camera frame embedded inline when one fell within the prior 100 ms window, and any 1 kHz position-stream samples from that window collected as a nested list.

25 parquet files across builds spanning 2026-05 through 2026-07. Build metadata (job name, profile, start/end time) is denormalized into every row, so each file is self-sufficient for ML — no joins needed.

from datasets import load_dataset
ds = load_dataset("ppak10/Inova-Mk1-Telemetry", split="train")
row = ds[0]
# row["frame_chamber"]            → PIL.Image.Image (or None)
# row["bedmatrix"]                → {width, height, values (768 °C floats), path} (or None)
# row["powderBed.temp.current"]   → float (°C)
# row["positions.position.z2"]    → float (µm)
# row["position_hf_burst"]        → list of {ts_offset_ms, x, y, z1, z2, r, has_homed}

Full image bytes are embedded in each row — no separate frame download. Filter to rows with a particular frame kind:

with_chamber = ds.filter(lambda r: r["frame_chamber"] is not None)

peregrine config — defect-labeled layers

A second, sparse config shaped like the ORNL Peregrine dataset: one row per layer that carries a defect label, for Peregrine-style transfer learning. Only labeled layers are recorded, so it's small and defect-focused (vs. the dense per-tick ticks config).

peregrine = load_dataset("ppak10/Agentic-SLS-Telemetry", "peregrine", split="train")
row = peregrine[0]
# row["image_after_powder"]  → JPEG bytes, post-recoat chamber still (Peregrine after_powder)
# row["image_after_melt"]    → JPEG bytes, post-scan chamber still   (Peregrine after_melt)
# row["part_mask"]           → PNG bytes, galvo scan mask            (Peregrine part_ids)
# row["labels"]              → list of {class, bbox [x0,y0,x1,y1] normalized, polarity}
# row["has_debris"], ...     → bool per class (PEREGRINE_ALL_CLASSES)
# row["printBed_temp"], row["laser_power_w"], ...  → process scalars at the layer

Images are raw encoded bytes (binary, like Peregrine's image_after_*) — decode with PIL.Image.open(io.BytesIO(...)). Labels are bbox-level (from the live defect detector / operator verdicts), not per-pixel masks; polarity is positive (defect present) or negative (operator marked the alert a false positive). One parquet per build (data/peregrine/{build:03d}.parquet), so a new build never rewrites existing files.

Timelapse previews

Per-build layer-by-layer timelapse GIFs (one frame per detected print layer, 25 fps, ≤ 12 s). Below is the composite (chamber | thermal | galvo) for build 012, the first full Layers-phase run:

[image]

_{Galvo (scan-mirror) trace for build 012 — Hex Coasters, 221 layers. Each frame is the last captured view of one print layer just before the next powder spread. Full composite (chamber | thermal | galvo) at previews/012/timelapse_composite.gif.}

Individual-kind GIFs (timelapse_chamber.gif, timelapse_thermal.gif) and full-speed MP4 previews are also available under previews/{build_id:03d}/. See the Previews section below.

Row shape

Each row is one moment in time (a single 10 Hz tick). ~80 columns:

Null frames

frame_* columns are null when no frame of that kind was captured within the 100 ms tick window — this preserves the "is this image fresh?" signal. Forward-fill for display or training:

import polars as pl
df = pl.read_parquet("data/ticks/026.parquet")
df = df.with_columns(
    pl.col("frame_chamber").forward_fill(),
    pl.col("frame_thermal").forward_fill(),
    pl.col("frame_galvo").forward_fill(),
)

Build inventory

25 builds total. Very short failed-heating runs (2–24 ticks) produce a parquet but represent only seconds of recording. The timelapse column shows the layer-by-layer composite GIF (chamber | thermal | galvo) for builds where the printer reached the sintering phase (z2 > 0).

¹ Build 041 was recovered from the NVMe spool after the recorder's Postgres database was reset; job_name is a placeholder.

Files

data/ticks/{build_id:03d}.parquet      # one file per build, zero-padded (001–043)

previews/{build_id:03d}/
  chamber.mp4                          # optical — real-time 10 fps, forward-filled
  thermal.mp4                          # IR bed heatmap (from bedmatrix; inferno, fixed 20–200 °C)
  galvo.mp4                            # scan-mirror trace
  composite.mp4                        # 1×3 panel: chamber | thermal | galvo
  timelapse_chamber.gif                # layer-by-layer, 25 fps, ≤ 12 s
  timelapse_thermal.gif
  timelapse_galvo.gif
  timelapse_composite.gif              # 1×3 panel GIF

The HF glob data/ticks/*.parquet loads the full dataset across all builds. Builds that never started printing (z2 = 0 throughout) produce a timelapse GIF with zero or one frame.

Previews

Real-time MP4s

Four MP4s per build under previews/{build_id:03d}/: chamber.mp4, thermal.mp4, galvo.mp4, and composite.mp4. Playback at 10 fps (the tick rate), so video duration equals build wall-clock time — build 026 → ~9h54m of video. Null frames are forward-filled for viewing continuity.

Timelapse GIFs

Four animated GIFs per build sampled one frame per detected print layer:

• —Layer detection: positions.position.z2 quantized in 100 µm buckets. Only z2 > 0 rows are used, so pre-print heating is automatically excluded.
• —Representative frame: the last non-null frame within each z2 bucket — the most-recent view of the layer just before recoating begins.
• —Playback: 25 fps, capped at 300 frames (12 s max). Null-frame levels are forward-filled within the GIF.
• —Canvas: 240 px height (half the MP4 canvas) for web-friendly file sizes.

Both previews are sourced from this dataset's own parquet (not the recorder's raw frame files), so they see only the per-tick-attached frame subset (~36% chamber, ~37% galvo attachment rate). The thermal panel is rendered from the numeric bedmatrix IR grid — inferno colormap over a fixed 20–200 °C range, so the same color means the same temperature across every build — rather than the legacy frame_thermal GIF. Builds 001/002/012 predate the bedmatrix stream and fall back to the old GIF.

What's lost vs the raw exporter format

This dataset is tick-anchored, not a strict outer join. Three things are reduced compared to the upstream flat exports:

• —Between-tick position resolution: the 1 kHz position_hf stream is preserved during motion via position_hf_burst, but not as standalone rows between ticks. For the raw 1 kHz timeline fall back to the upstream position_hf/{build_id}.parquet.
• —Sparse events without telemetry: no row exists for a camera frame or events record that doesn't land near a 10 Hz tick. This dataset represents "state at each tick", not "every recorded event".
• —`plotter_commands`: not surfaced — historical builds predate this recorder feature and the table is empty.

Joining back to Inova-Mk1-Database

Each row carries print_profile_name — the string the printer was running when the build started, taken from the build_start event payload. To recover the matching PrintProfile.Id UUID, join against `ppak10/Inova-Mk1-Database`:

import json
from pathlib import Path

profiles_dir = Path("Inova-Mk1-Database/source/PrintProfiles")
name_to_id = {
    json.load(p.open())["Name"]: json.load(p.open())["Id"]
    for p in profiles_dir.glob("*.json")
}
ds = ds.map(lambda r: {**r, "print_profile_id": name_to_id.get(r["print_profile_name"])})

Upstream

Raw data lives in `Agentic-Additive-Manufacturing-Process-Optimization`. Its scripts/export.py produces flat JSONL/parquet files under data/exports/. This dataset reads from those directly — no live database connection required.

Regenerate

# Full ETL (all builds)
uv run scripts/ticks/01_extract.py

# Specific builds only
uv run scripts/ticks/01_extract.py 26 28

# MP4 previews
uv run scripts/previews/01_render.py          # all builds
uv run scripts/previews/01_render.py 26 28    # specific builds

# Timelapse GIFs
uv run scripts/previews/02_timelapse.py          # all builds
uv run scripts/previews/02_timelapse.py 26 28    # specific builds