Darkweb007/cuda-kernels-live

CUDA Kernels — live on ZeroGPU

Runs real, JIT-compiled CUDA kernels on a free Hugging Face ZeroGPU allocation:

1. 1.Sliding-window attention — a from-scratch kernel (online softmax, Longformer-style local window), compiled at request time with torch.utils.cpp_extension.load_inline, benchmarked against dense masked PyTorch attention.
2. 2.Kernel fusion compiler — y = gelu(x*w + b) fused from 3 elementwise ops into 1 generated CUDA kernel by `fusion_compiler`, compiled and run, benchmarked against the naive 3-kernel-launch version.

Companion Spaces/repos:

• —long-context-attention-kernels — the production tiled kernel this demo's simplified version is based on
• —cuda-fusion-compiler — the fusion compiler used in tab 2

Why ZeroGPU

ZeroGPU attaches a GPU to the process only for the duration of a function decorated with @spaces.GPU, so all CUDA work here (JIT compile + kernel launch + benchmark) happens inside those functions. First call per session compiles the kernel (a few seconds); later calls reuse the on-disk build cache.

Local run (needs your own CUDA GPU)

pip install -r requirements.txt
python app.py

Locally spaces.GPU is a no-op decorator (the spaces package falls back gracefully off of a ZeroGPU Space), so this also runs on any machine with a CUDA-capable GPU and the toolchain installed.