yitongl/sparse_quant_exp

1# Standalone Inference Helper2 3This folder contains a portable inference helper for:4 5`sfp4_v4_sparse09_hpo_on_ours_p_init2050_1n_interactive/checkpoint-700`6 7It is not a full vendored copy of Wan or FastVideo.  It contains the sparse FP48backend overlay and a runner that can be applied to a FastVideo checkout or9installation so the uploaded checkpoint can be used for normal inference.10 11## Contents12 13- `run_inference.py`: downloads/loads `transformer/diffusion_pytorch_model.safetensors` from `yitongl/sparse_quant_exp` and runs `VideoGenerator`.14- `run.sh`: convenience wrapper that installs the overlay into `FASTVIDEO_ROOT` and then runs `run_inference.py`.15- `install_overlay.py`: copies the bundled sparse FP4 backend files into a FastVideo checkout/install.16- `overlay_files/`: exact runtime source files needed by `SPARSE_FP4_OURS_P_ATTN`.17- `training_attention_settings.json`: structured settings for the uploaded checkpoint.18 19## Expected Environment20 21- A working FastVideo Python environment.22- FastVideo dependencies installed, including PyTorch, Triton, safetensors, and23  Hugging Face Hub.24- Access to the base model `Wan-AI/Wan2.1-T2V-1.3B-Diffusers`.25- A CUDA GPU supported by the custom Triton kernels.26 27## Usage28 29From a machine with this HF repo downloaded:30 31```bash32export FASTVIDEO_ROOT=/path/to/FastVideo33bash standalone_inference/run.sh \34  --output-path outputs/sfp4_checkpoint_700 \35  --seed 100036```37 38The script sets:39 40```bash41FASTVIDEO_ATTENTION_BACKEND=SPARSE_FP4_OURS_P_ATTN42FASTVIDEO_SPARSE_FP4_USE_HIGH_PREC_O=143```44 45and downloads the uploaded checkpoint-700 transformer weights unless `--weights`46is provided.47 48To use a local safetensors file:49 50```bash51export FASTVIDEO_ROOT=/path/to/FastVideo52bash standalone_inference/run.sh \53  --weights /path/to/diffusion_pytorch_model.safetensors \54  --prompt "your prompt"55```56 57## Attention Semantics58 59- Self-attention uses `SPARSE_FP4_OURS_P_ATTN`.60- Q/K/V use FP4 fake quantization with STE.61- VSA tile size is `4 x 4 x 4 = 64` tokens.62- Selected sparse tiles use group-local P quantization in the Triton kernel.63- Dropped tiles use tile mean compensation.64- Cross-attention falls back to dense SDPA and is not sparse/FP4.65 66## Checkpoint67 68The current HF `main` transformer file is checkpoint-700:69 70`transformer/diffusion_pytorch_model.safetensors`71 72Local SHA256 used when preparing this helper:73 74`4595ca81ea7085c15ccf14b738aa9c0fdf2d2786641f49b55e0bc0e99bf042d2`75