CoolFace
Modelpublic

FlyCockpit/GLM-5.2-NVFP4-AQLM-Abliterated-Vision

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face
0likes35downloads
SETUP.md16 linesDownload Raw Back to code
1# New-box setup2 3The vLLM changes now live in the fork **github.com/jarrelscy/vllm-glm52-sm120**4(branch `glm52-sm120`) — everything is committed, there are no patches to apply.5 61. `bash code/run_rtx6000.sh [1m|500k|250k]` — does everything (clones the fork,7   builds the venv with `VLLM_USE_PRECOMPILED=1`, installs flashinfer git-main for8   the SM120 sparse-MLA path, sets up the JIT toolchain, fetches weights, serves).9   - Or use the fork's Docker image (RTX PRO 6000):10     `docker build -f Dockerfile.glm52-sm120 -t glm52-sm120 .` then11     `docker run --gpus all --ipc=host -p 8001:8001 -v <weights>:/models/1m -e PARALLEL=tp4-1m-mtp glm52-sm120`12     (see the Dockerfile header for all `PARALLEL` modes: pp4-1m, tp4-1m-mtp, tp2pp2, …).132. Verify with `code/tools/verify_sm120.py` against the golden bundle14   (upload it separately or re-capture via `code/tools/capture_golden.py`).153. Full runbook: `code/PLAN.md`; history: `code/glm52-RUNLOG.md`.16