nc-murray/spectrogram-reconstruction

specrec — Spectrogram-to-Audio Reconstruction

In May 2026, the NTSB inadvertently published a spectrogram image in a public investigation docket. Members of the public used signal processing tools to invert that image back into approximate audio. This project is built to understand and demonstrate that exact mechanism — using programmatically-generated synthetic audio as the test case — and to answer a question the media coverage skipped: how accurate is the reconstruction, quantitatively?

How it works

Griffin-Lim phase reconstruction. A spectrogram encodes magnitude only — the STFT phase is discarded. The Griffin-Lim algorithm (1984) iteratively estimates consistent phases by cycling between the time and frequency domains: invert the spectrogram to a waveform, re-compute its STFT, replace the magnitudes with the known target, repeat. More iterations produce better phase estimates. librosa.griffinlim() provides the implementation.

The colormap inversion problem. A spectrogram image published as a PNG encodes magnitude values through a matplotlib colormap — each pixel's RGB triple represents a normalised dB value. To reverse this, specrec builds a 256-entry lookup table for each supported colormap by sampling it uniformly, then maps every pixel to its nearest LUT entry via Euclidean distance in RGB space using a k-d tree (scipy.spatial.cKDTree). This nearest-neighbour search is purely mathematical — no model required, no external data.

Installation

pip install -e ".[dev]"

Requires Python 3.10+. No GPU required.

Quick start

# Generate a test signal and its spectrogram image
specrec test --type tone --output-dir ./demo

# Reconstruct audio from the image
specrec reconstruct \
  --input  demo/tone_440hz_viridis.png \
  --output demo/tone_reconstructed.wav \
  --colormap viridis --freq-max 11025 --duration 2.0

# Compare reconstruction against the original
specrec evaluate \
  --input     demo/tone_440hz_viridis.png \
  --reference demo/tone_440hz.wav \
  --colormap viridis --freq-max 11025

# Run the full synthetic demo (both signals, all metrics, all plots)
specrec demo --output-dir examples/synthetic_demo

CLI reference

specrec reconstruct

--input      IMAGE_PATH     Spectrogram PNG or JPEG
--output     AUDIO_PATH     Output WAV file
--colormap   NAME           viridis, jet, magma, plasma, inferno, Greys
                            (auto-detected if omitted)
--freq-max   HZ             Highest frequency in the image  [default: 11025]
--db-min     DB             dB floor used when rendering   [default: -80]
--db-max     DB             dB ceiling                     [default: 0]
--sr         HZ             Sample rate                    [default: 22050]
--n-fft      N              FFT size                       [default: 2048]
--hop        N              Hop length                     [default: 512]
--n-iter     N              Griffin-Lim iterations         [default: 60]
--duration   SECONDS        Original duration (resizes time axis when provided)

specrec test

Generates synthetic audio and its spectrogram PNG for controlled experiments.

--type       tone|speech_like
--output-dir DIR
--colormap   NAME

specrec evaluate

Reconstructs from an image and prints spectral convergence and SNR against a reference WAV.

specrec demo

Runs the full round-trip on both test signals, generates all plots, and writes accuracy_report.json.

Accuracy characterisation

The round-trip test: generate known audio → render to spectrogram PNG → parse image → reconstruct → compare.

Spectral convergence = ‖S_orig − S_recon‖_F / ‖S_orig‖_F (Frobenius norm ratio; 0 = perfect, 1 = unrelated noise, both amplitude-normalised before comparison).

SNR = signal-to-noise ratio in dB after amplitude alignment. Note: Griffin-Lim finds a phase-consistent solution, not the original phase. The reconstructed signal has the correct frequency content but arbitrary phase, so time-domain SNR is low even when spectral structure is well-preserved. Spectral convergence is the more meaningful metric.

Results

Spectral convergence decreases monotonically with iterations. The pure tone reaches ~0.376 at 100 iterations; the speech-like signal plateaus near 0.470, reflecting greater phase ambiguity in multi-component signals.

Colormap sensitivity (n\_iter = 60)

Reconstruction quality is effectively independent of colormap choice — as long as the colormap is known. All four produce spectral convergence within 0.001 of each other.

Plots

Generated by specrec demo:

Limitations

Project structure

src/specrec/
├── cli.py            # Click entry point
├── synthesizer.py    # Synthetic audio generation, audio → PNG
├── image_parser.py   # Colormap LUT inversion, PNG → magnitude array
├── reconstructor.py  # Griffin-Lim pipeline, magnitude → WAV
├── evaluator.py      # Round-trip accuracy metrics
└── visualizer.py     # Comparison and accuracy plots

tests/
├── test_reconstructor.py
└── test_evaluator.py

notebooks/
└── walkthrough.ipynb   # Step-by-step visual demo

examples/synthetic_demo/   # Generated by `specrec demo`

Development

pytest                        # run tests
ruff check src/ tests/        # lint
specrec demo                  # regenerate all demo outputs
jupyter lab notebooks/        # open interactive walkthrough

References

• —Griffin, D. & Lim, J. (1984). Signal estimation from modified short-time Fourier transform. IEEE Transactions on Acoustics, Speech, and Signal Processing, 32(2), 236–243.
• —McFee, B. et al. (2015). librosa: Audio and music signal analysis in Python. Proceedings of the 14th Python in Science Conference. librosa.org