clark-labs/clark-swin2sr-lightweight-x2-1.58bit

Clark Swin2SR ×2 Ternary — a 320 KB full-surface-ternary browser AI upscaler

This is a quality-oriented ternary Swin2SR model for 2× image super-resolution. It runs entirely on the user's device: no server inference, no image upload, and no watermark.

Latest checkpoint (2026-08-03): step-27500 text-robustness retrain. Trained on a broadened 12-way degradation battery (bicubic/lanczos/bilinear/area downscales, JPEG chroma, WebP, blur, noise, DPR rescale, sharpening halos, gamma/color shift) plus a new 40k dense-text corpus (22 fonts, 7 languages, code and UI copy). It beats the previous ternary release on all 12 realistic degradations (mean +0.29 dB PSNR, worst case +0.13 dB, PSNR and edge-PSNR) with OCR legibility on par or better. The trade: clean-bicubic Set5 mean is 34.47 dB versus 35.30 dB for the previous release — see Quality for the full honest picture.

The model is deliberately not described as “exactly 1.58-bit.” It follows the same practical design used by quality-oriented low-bit image models:

• —204 matrix and convolution tensors are ternary {−1, 0, +1} — including the first and final image-I/O convolutions;
• —only layernorm vectors and biases remain FP16 (280 small raw tensors);
• —97.24% of all parameters are ternary-quantized (absmean, group size 60);
• —the complete lossless native browser package is 320,318 bytes, or 2.549 bits per model parameter including its manifest.

The public repository name is retained for compatibility with existing links, but the storage accounting below is the authoritative description.

See the difference

Each comparison starts from the same low-resolution Set5 image. “Before” is ordinary bicubic enlargement. “After” is this model. The cards use a 100% detail crop and add no post-model sharpening.

[image]

[image]

[image]

▶ Try the live demo →

The Space uses WebGPU when available and falls back to WebAssembly, with a worker pool that runs tiles in parallel across up to four ONNX sessions. For the same private on-device workflow inside a product-demo editor, try BackdropKit.

Quality

Set5 ×2

Full-image RGB PSNR across all five Set5 images:

The complete per-image receipt is in `examples/set5-x2-results.json`. Note the previous bonsai release scored 35.30 dB on this clean-bicubic benchmark; the step-27500 checkpoint trades 0.83 dB of clean-bicubic fidelity for the robustness gains below, which matter more on real screenshots that arrive compressed, blurred, rescaled, or gamma-shifted.

Realistic UI degradation battery (this release's headline metric)

25 real UI screenshots × 12 realistic degradations, full-image RGB PSNR and edge-PSNR versus ground truth, plus OCR legibility:

The step-27500 checkpoint wins PSNR and edge-PSNR on all 12 degradations (worst case +0.12 dB). A larger FP16-q4 reference still scores higher on pure fidelity; this model's value is text robustness at 320 KB. Full receipts: `technical/candidate-ledger.json`.

Locked development bundle

The previous release scored 34.31 dB RGB PSNR / 37.87 dB edge PSNR on the locked 3,040-image development bundle (product imagery, real UI captures, synthetic UI text in 204 languages). That bundle is currently being rebuilt, so this checkpoint has not been re-scored on it; the realistic battery above is the release gate for this update.

Honest storage accounting

The distinction matters. log₂(3) ≈ 1.585 describes the information in one ternary code; it does not include group scales, raw parameters, alignment, or a runtime manifest. This card reports each layer separately. The portable ONNX is a dense FP32 export of the hard-materialized ternary state (weights are exact ternary values stored as floats), which is why its gzip is larger than the packed native package.

Runtime verification

The exact release state passed:

• —dense-state → ONNX numerical conformance at 133.1 dB PSNR versus PyTorch;
• —packed native-artifact conformance at 76.1 dB PSNR (pass);
• —12-degradation realistic-UI battery, PSNR + edge-PSNR + OCR (receipts in technical/).

At a 64×64 input (128×128 output) in the release browser harness, warm ONNX inference measured 30.4 ms on WebGPU and 377.3 ms on WASM (previous release; the graph shape is unchanged so compute cost is the same). The Space now runs tiles through a pool of up to four parallel ONNX sessions, cutting wall-clock time roughly 3–4× on multi-core machines. Performance varies with browser, hardware, image size, tiling, and thermal state.

Files

• —model.safetensors — standard Transformers/PyTorch weights (dense-hard export: ternary weights materialized as exact −1/0/+1 floats).
• —onnx/model.onnx.gz — portable browser model; WebGPU with WASM fallback.
• —onnx/model.onnx — the same graph without transport compression.
• —native/swin2sr-ternary-bonsai-x2.bdk.gz — the lossless 320 KB packed Clark browser-runtime package (filename retained for link compatibility).
• —technical/quantization.json — exact artifact hashes, storage, and quality receipts.

Quick start

Python with Transformers

import torch
from PIL import Image
from transformers import AutoImageProcessor, Swin2SRForImageSuperResolution

model_id = "clark-labs/clark-swin2sr-lightweight-x2-1.58bit"
processor = AutoImageProcessor.from_pretrained(model_id)
model = Swin2SRForImageSuperResolution.from_pretrained(model_id).eval()

image = Image.open("input.png").convert("RGB")
inputs = processor(image, return_tensors="pt")

with torch.inference_mode():
    output = model(**inputs).reconstruction

output = output[:, :, : image.height * 2, : image.width * 2]

Browser with ONNX Runtime Web

<script type="module">
import * as ort from "https://cdn.jsdelivr.net/npm/onnxruntime-web@1.27.0/dist/ort.webgpu.min.mjs";

ort.env.wasm.wasmPaths =
  "https://cdn.jsdelivr.net/npm/onnxruntime-web@1.27.0/dist/";

const response = await fetch(
  "https://huggingface.co/clark-labs/clark-swin2sr-lightweight-x2-1.58bit/resolve/main/onnx/model.onnx.gz"
);
if (!response.ok) throw new Error("Model download failed");

const modelBytes = await new Response(
  response.body.pipeThrough(new DecompressionStream("gzip"))
).arrayBuffer();

const providers = "gpu" in navigator ? ["webgpu", "wasm"] : ["wasm"];
const session = await ort.InferenceSession.create(modelBytes, {
  executionProviders: providers,
});

// `pixels` is planar RGB Float32 data in [0, 1]. Pad dimensions to multiples
// of eight, run `pixel_values`, then crop `reconstruction` to 2× the original.
const input = new ort.Tensor(
  "float32",
  pixels,
  [1, 3, paddedHeight, paddedWidth]
);
const { reconstruction } = await session.run({ pixel_values: input });
</script>

What it is good at

• —Zoomed product screenshots and interface details
• —Small labels, icons, controls, and clean UI edges — including text that survived JPEG/WebP compression, blur, noise, or display rescaling
• —Product thumbnails, illustrations, and natural images
• —Private, offline, or zero-server-cost image pipelines

Limitations

• —It performs fixed 2× super-resolution.
• —It is restorative, not generative: it cannot recover information that never existed in a severely blurred or compressed source.
• —On clean bicubic-downscaled benchmarks (Set5) it scores 0.83 dB below the previous bonsai release; its gains are on real-world degraded inputs.
• —Benchmark conventions differ. Scores here are full-image RGB PSNR, not luminance-only scores with cropped borders.
• —The compact packed runtime is Clark-specific; use the ONNX file for standard cross-browser integration.

Lineage and credits

• —Architecture: Swin2SR, Conde et al.
• —Base weights: `caidas/swin2SR-lightweight-x2-64`
• —Set5 data: `eugenesiow/Set5`
• —Quantization, training, browser packaging: Clark Labs

Citation

@misc{clark2026swin2srternary,
  title  = {Clark Swin2SR Lightweight x2 Ternary:
            Heterogeneous-Precision Super-Resolution for the Browser},
  author = {Clark Labs},
  year   = {2026},
  url    = {https://huggingface.co/clark-labs/clark-swin2sr-lightweight-x2-1.58bit}
}

Apache 2.0—use it anywhere, including commercially.