Darkweb007/rope-swiglu-kernels

RoPE + Fused RMSNorm / SwiGLU Kernels -- interactive showcase

Companion Space for rope-swiglu-kernels, three CUDA kernels covering RoPE, RMSNorm, and SwiGLU -- the non-attention building blocks of a LLaMA/Mistral/Qwen-style transformer layer.

This Space is deliberately CPU-only: no GPU, no nvcc, no ninja, no live CUDA compilation. It runs the project's real pure-PyTorch reference implementation (the exact code path the actual .cu kernels are checked against in tests/test_correctness.py) and visualizes it interactively -- rotation angles for RoPE, the normalizing effect of RMSNorm, and the SwiGLU activation curve. The .cu kernel source is included as a read-only preview under the "Kernels" tab.

No GPU/toolchain dependency means nothing here can break the way live JIT kernel compilation can -- see the GitHub repo for the actual CUDA source and build instructions if you want to compile and run the real kernels on a CUDA machine.