Modelpublic

BBuf/helios-base-modelopt-fp8-sglang-transformer

sourceHugging Faceupdated 5mo agoView on Hugging Face

0likes8downloads

Model Card

Helios-Base ModelOpt FP8 SGLang Transformer

This repository contains the SGLang-ready ModelOpt FP8 transformer override for BestWishYsh/Helios-Base. It is not a standalone pipeline checkpoint. Keep the BF16 base model as --model-path and pass this repo as --transformer-path.

Usage

bash

sglang generate \
  --model-path BestWishYsh/Helios-Base \
  --model-id Helios-Base \
  --transformer-path BBuf/helios-base-modelopt-fp8-sglang-transformer \
  --prompt "A curious raccoon" \
  --width 640 \
  --height 384 \
  --num-frames 33 \
  --seed 42 \
  --save-output

For local mirrors of BestWishYsh/Helios-Base, keep --model-id Helios-Base so SGLang selects the native Helios integration.

Validation

Validation was run on one H100 GPU with only rank 0 visible.

Run	Denoise	E2E	Peak reserved	Notes
BF16 baseline	95.80 s	96.85 s	59.31 GiB	640x384, 33 frames, 50 steps, seed 42
ModelOpt FP8	83.72 s	84.77 s	47.36 GiB	same command plus `--transformer-path`

The FP8 run was 1.14x faster on denoising and used 11.94 GiB less peak reserved GPU memory in this setup.

Validation videos, frame contact sheets, perf JSON files, profiler traces, and the kernel summary are available under the validation/ directory in this repo.

Profiler Snapshot

Torch profiler full-stage traces were collected with 5 inference steps at the same 640x384, 33-frame shape.

Run	CUDA kernel time	GEMM/Linear	Attention	Elementwise/Cast/Reduce	Conv/VAE	FP8 quantize overhead
BF16	9996.4 ms	34.97%	14.14%	41.44%	4.81%	0.00%
FP8	8980.3 ms	26.62%	15.35%	45.61%	5.35%	1.95%

Conversion Notes

The SGLang converter uses the Helios-specific BF16 fallback profile for quality stability. The fallback keeps patch embeddings, conditioning/output projections, and the first/last three transformer blocks in BF16 while materializing eligible linear weights as float8_e4m3fn.

Converter output summary:

json

{
  "added_scale_tensors": 680,
  "bf16_fallback_weights": 70,
  "output_shards": 3,
  "preserved_ignored_weights": 200,
  "quantized_weights": 340
}