Phips/NERVE

NERVE

Norm-free Efficient Restoration for Various Edge devices

A lightweight super-resolution architecture that is simple to train and fast to run. Released models are verified with traiNNer-redux (training) and ONNX Runtime + ncnn (inference). The op set is standard, so other runtimes (CoreML, TensorRT, DirectML, ...) are expected to work too — but they are not tested here.

Part of the BODY suite by Philip Hofmann — networks built to be real-world usable first, not benchmark-chasing. See Relation to HEART for which one to pick.

NERVE links: models & configs · try it online (ZeroGPU) · convert a checkpoint · local app · train your own (Colab)

Short version: copy nerve_arch.py into traiNNer-redux, train with the sample configs, then run the .safetensors, .onnx, or .ncnn models. One architecture, one size, no fused/unfused pairs. Details below.

[image] Input (as-is) | bicubic x4 | NERVE 4x release | NERVE 4x OTF GAN.

[image] [image]

Why NERVE?

There are strong and established SR networks already. This is the honest case for when NERVE is worth your time — and when it is not.

This project comes out of a trainer's frustration rather than a benchmark goal. After training hundreds of SISR models (see Phhofm/models), the recurring annoyance was networks that felt like paper networks: they chase leaderboard metrics with techniques that leave you juggling an unfused and a fused checkpoint, or whose dynamic-ONNX conversion is fragile. This network is the opposite of that — built so a trainer can pick it up, train it, and convert it to dynamic ONNX without headaches.

Strengths of NERVE

• —Real-world usability over metric chasing. Trained on a CC0 dataset with the full Real-ESRGAN OTF degradation pipeline; the OTF GAN model restores degraded images, not just clean bicubic downsamples.
• —One file, one variant. A single readable architecture file and one model size. No S/M/L matrix, no fused-vs-unfused checkpoints, nothing to fuse before deploy — what you train is what you ship.
• —Export just works. Dynamic-shape ONNX (verified, ORT parity ~1e-5) and a clean ncnn conversion. Standard ops only (Conv / Add / ReLU / DepthToSpace / Resize).
• —Boring, stable training. Norm-free, bf16 AMP, no gradient clipping, a single Charbonnier loss for the base stage.
• —Small and fast. ~1.8M params — SPAN-class size but conv-only, suited to mobile, edge, web and game engines.
• —Maintainable for years. One op set, no exotic dependencies, ~85-line arch.

When to pick something else. If the goal is maximum PSNR/LPIPS on academic benchmarks, or a large transformer, NERVE is not aiming there — use HEART, HAT, ATD, DAT2, or one of the community networks below. NERVE aims at simplicity, deployability and hassle-free training.

Credit where it is due. None of this exists in a vacuum. SISR has many strong architectures and generous community members, for example HAT, ATD and DAT2, the community network work of umzi and Kim2091, fdat by sharekhan, and the-database (traiNNer-redux, its benchmarks and its models). This list is far from complete — apologies to anyone not named; the SISR community has a great many people contributing a great deal.

Quickstart

As a user (just want to upscale / restore images)

Pick a model and run it:

• —ONNX Runtime (verified): use onnx/*.onnx (dynamic H/W). TensorRT / DirectML should work too (same op set) but are untested here.
• —ncnn (verified, fp16): ncnn/*.ncnn.param + *.ncnn.bin. Feed images with Mat.from_pixels:

  import ncnn
  net = ncnn.Net(); net.load_param("4x_NERVE_fp16.ncnn.param"); net.load_model("4x_NERVE_fp16.ncnn.bin")
  ex = net.create_extractor()
  ex.input("in0", ncnn.Mat.from_pixels(bgr_uint8, ncnn.Mat.PixelType.PIXEL_BGR2RGB, w, h))
  out = ex.extract("out0")[1]   # CHW float, 4x

(Use from_pixels; the ncnn Python binding's raw-array constructor misbehaves on large inputs. Outputs are fp16, so expect small deviations at hard edges.)

• —chaiNNer: load the onnx/*.onnx files directly. Loading the .safetensors via spandrel needs NERVE registered in your chaiNNer/spandrel build (not upstream yet).
• —Input: RGB, any size. Output: 2x or 4x.

As a trainer (want to train your own)

NERVE is a normal traiNNer-redux architecture — no special steps.

# 1. Add the architecture + its ICNR helper (auto-registers: traiNNer scans *_arch.py)
cp nerve_arch.py <traiNNer-redux>/traiNNer/archs/
cp icnr.py       <traiNNer-redux>/traiNNer/utils/   # ICNR init helper (not in upstream yet)

# 2. Copy a training config and point it at your data
cp configs/4x_NERVE_release.yml <traiNNer-redux>/options/
#    -> edit dataroot_gt / dataroot_lq in that file

# 3. Train
cd <traiNNer-redux>
python train.py -opt options/4x_NERVE_release.yml --auto_resume

# 4. Export to optimized dynamic ONNX (official converter)
cp configs/4x_NERVE_onnx.yml <traiNNer-redux>/options/
#    -> set path.pretrain_network_g to your trained checkpoint
python convert_to_onnx.py -opt options/4x_NERVE_onnx.yml

That is the whole workflow — 2 small files plus a config. The net is one file, the config is one file, the checkpoint is one file — no fusion step, no paired fused/unfused weights. (icnr.py is the only piece not in upstream traiNNer-redux yet; it is the ICNR initializer for the PixelShuffle head.)

• —4x_NERVE_onnx.yml exports dynamic-shape ONNX (opset 20) and can emit fp32, fp16, or bf16 (onnx.dtype), with OnnxSlim optimization and a built-in PyTorch-vs-ONNX verify pass.
• —Prefer no extra dependencies? scripts/export_dynamic.py is a tiny standalone exporter (torch + onnxruntime only).

Start from a released pretrain (skip the expensive early training) by setting path.pretrain_network_g in your training config to a file from models/:

Training tips (all NERVE models follow these):

• —bf16 AMP is fine; no gradient clipping needed (norm-free net).
• —--auto_resume is recommended for long runs.
• —For real-world input, follow the chain: clean release -> OTF fidelity -> OTF GAN.

Models

models/*.safetensors are plain EMA state dicts (no wrapper) — load them with nerve_arch.nerve(scale=N).

All released models (2x/4x release, 4x OTF fidelity, 4x OTF GAN) are final.

Pretrain chain (how the models relate):

2x_NERVE_release ──→ 4x_NERVE_release ──→ OTF fidelity ──→ OTF GAN

The release models are the shared starting point for anyone training NERVE: warm-start from them (2x → 4x, or release → OTF fidelity → OTF GAN) to skip the expensive early training.

Examples

Real images are fed in as-is (not downscaled) and upscaled 4x. For each image there are two comparisons:

• —examples/*_compare.png — whole image at full resolution: input | bicubic x4 | NERVE 4x release | NERVE 4x OTF GAN.
• —examples/*_1to1.png — a 1:1 native-pixel center crop.

The bicubic column is the baseline; the OTF GAN column is the released showcase model. The source inputs are in examples/inputs/.

• —examples/realphoto_compare.png (street scene)
• —examples/ani_compare.png (anime)
• —examples/manga_compare.png (manga)
• —examples/text_compare.png (text)
• —examples/noisy_compare.png (noisy input)
• —examples/stablediffusion_compare.png (AI-generated)

Architecture

5x5 conv stem → 24 × (Conv-ReLU-Conv + residual) → PixelShuffle → + bicubic(x)

• —~1.8M params, pure convolution — no attention, no normalization, no gating, no positional encoding, no reparameterization.
• —A bicubic input residual means the net only learns what bicubic upscaling misses — that's why it stays small and trains fast.
• —ICNR-initialized PixelShuffle head (checkerboard-free by construction).
• —The op graph is trivial: Conv, Add, ReLU, DepthToSpace, Resize. That's the entire reason it exports to dynamic ONNX and converts to ncnn (verified) without custom ops or fused/unfused pairs; CoreML is expected but untested.

What we actually tested

Every design choice here came from a measured ablation, not a guess. The full development log — including dead ends and raw numbers — is in `docs/ABLATIONS.md`. Highlights:

• —Normalization: removing LayerNorm beat the normed baseline by +1.15 dB at ~2x the training speed -> norm-free.
• —Depth/width sweep + capacity scaling: quality improved with size; the sweep put the GAN quality knee at 1.80M (0.69M -> 1.80M: lpips -0.015; 1.80M -> 2.39M: only -0.002 more).
• —Dilation: a dilated-conv A/B under the OTF recipe was null -> plain convs.
• —Checkerboard: diagnosed a PixelShuffle phase-lock, fixed with ICNR plus keeping aliased (nearest-exact) LR. An anti-aliased-LR + sub-pixel-jitter variant made thin lines worse and was reverted.
• —OTF/GAN capability: the small net trained productively under the full Real-ESRGAN OTF GAN recipe (no collapse).
• —Runtime validation: every released .safetensors loads (strict) and runs; every .onnx loads in ONNX Runtime; the ncnn models were run at multiple input sizes up to 720x720 (2880x2880 output).
• —Deployability: dynamic-ONNX export + ORT parity and ncnn conversion were verified at every stage (the numbers are in the log).
• —Not tested here (community welcome): TensorRT / DirectML speed and fixed-shape static exports, CoreML, and on-device benchmarks. Train more variants, benchmark, and share — the arch is standard ops only.

Training data & license

All released NERVE models were trained only on `Phips/lucid-cc0-v2-hc-512`, a CC0 dataset — the released weights are clean to use commercially.

We did not optimize for benchmark leaderboards. If you want higher PSNR/LPIPS numbers, training on larger academic datasets (DIV2K, Flickr2K, LSDIR, ...) is expected to score better; NERVE is a normal architecture, train it on whatever you like.

Relation to HEART

NERVE and HEART are the two members of the BODY suite (by Philip Hofmann): same philosophy (real-world usability > benchmark scores), different tiers.

Rule of thumb: HEART for maximum quality, NERVE for maximum practicality. Both export to dynamic ONNX; NERVE additionally converts to ncnn cleanly because it has no attention ops.

Side-by-side comparisons on the same inputs (input | bicubic | both release models | both OTF GAN models), each with a 1:1 crop: examples/OST_009_nerve_vs_heart_* and examples/00003_nerve_vs_heart_* (Real-ESRGAN test set), plus examples/realphoto_nerve_vs_heart_* and examples/noisy_nerve_vs_heart_*.

Repository layout

nerve_arch.py                    the architecture (copy to traiNNer/archs/)
icnr.py                          ICNR init helper (copy to traiNNer/utils/; not in upstream yet)
configs/                         example training configs (+ 4x_NERVE_onnx.yml for ONNX export)
models/                          pretrained checkpoints (see table above)
onnx/                            dynamic fp32 ONNX (opset 20, onnxslim-optimized)
ncnn/                            ncnn fp16 param/bin
docs/ABLATIONS.md                the full development/ablation log (what we tested)
examples/                        real-image comparisons (input vs bicubic vs release vs GAN)
scripts/export_dynamic.py        checkpoint → dynamic ONNX exporter (+ ORT verify)
LICENSE                          Apache-2.0

Help wanted (community welcome)

The goal here is real-world usability — easy training, hassle-free dynamic ONNX, simple and robust to maintain — not leaderboard chasing. So NERVE's strength is simplicity for trainers; there's still plenty worth measuring. Pick anything below and open an issue/PR/discussion with your numbers.

Benchmarks (deliberately not chased here)

• —Train on an academic dataset (DIV2K / Flickr2K / LSDIR) and compare against SPAN, SRVGGNetCompact, HAT, Real-ESRGAN (PSNR / SSIM / LPIPS).
• —Perceptual metrics (LPIPS / TopIQ / MUSIQ) on real-world degraded images.

Speed & deployment (not benchmarked here)

• —TensorRT / DirectML latency + peak VRAM on the dynamic ONNX.
• —Static-shape ONNX (e.g. 1x3x256x256) vs dynamic: speed + memory.
• —fp16 vs fp32 ONNX.
• —ncnn on-device (Android / iOS): latency + memory.
• —CoreML conversion + on-device test.
• —Mobile CPU (XNNPACK / ORT Mobile / ncnn).

Models & code

• —More finetunes (illustration, manga, denoise, 1x restoration).
• —Architecture experiments — see `docs/ABLATIONS.md` for what we already tested (and rejected).

Small, measured contributions are welcome.

License

Apache-2.0 (this repository's code and pretrained weights).

Update — 2026-09-22: speed benchmark (honest numbers)

We ran a head-to-head inference benchmark on an RTX 3060 using the traiNNer-redux benchmark protocol (input 1x3x480x640, fp16, 50 runs), against the closest networks by parameter count:

NERVE does not win on speed. SPANS is ~2.1x faster at a similar parameter count, and compact is faster still. (PLKSRtiny is slower in PyTorch and falls apart in ONNX/TensorRT because its large-kernel/CCM ops don't map, so NERVE does beat it there.)

That is neither a surprise nor a problem for what NERVE is for. NERVE is not a benchmark-chasing network. It exists for ease of use and real-world usability: one variant, one file, no fused/unfused checkpoint pairs, and a checkpoint that converts to dynamic-shape ONNX, TensorRT and ncnn with no special handling. Speed is good enough, not the goal.

Practical notes from the benchmark:

• —TensorRT is the fastest runtime for NERVE (~2.5x over PyTorch), and it wants a static shape — that is where static exports actually matter.
• —In ONNX Runtime, static vs dynamic made no measurable difference.
• —fp16 ≈ bf16.
• —ncnn runs on the GPU via Vulkan (117 ms vs 5.4 s on CPU), but that is the Python binding; native ncnn on a phone is the intended path, not a desktop race.

Choose SPAN if you want maximum speed. Choose NERVE if you want the simplest thing to train, convert and ship.

Update — 2026-09-25: 2x OTF GAN (and a patch/batch result)

Added a 2x OTF GAN restoration model: warm-started from the 2x OTF fidelity chain and finetuned with a larger patch (`lq 128`) and batch (8) for 150k iterations, then validated against the previous 2x GAN (lq96/batch4, 150k) on the same OTF-degraded Urban100 x2 set, full-frame:

The larger patch + batch improved every metric, so this is the released 2x OTF GAN. (The tiled-vs-full-frame validation difference was measured and is negligible: ~0.007 dB PSNR.)