MBFSAITeam/cuda-runtime-bundle

1---2license: other3license_name: nvidia-cuda-cudnn-redistributable4license_link: https://docs.nvidia.com/cuda/eula/index.html5pretty_name: MBFS CUDA + TensorRT Runtime Bundle (cuda_v12 + cuda_v11 + trt_v12)6tags:7  - cuda8  - cudnn9  - tensorrt10  - onnxruntime11  - runtime-libraries12  - windows13---14 15# MBFS CUDA Runtime Bundle16 17Self-contained, **ABI-matched** CUDA execution stacks for running18[ONNX Runtime](https://onnxruntime.ai/) on Windows with the CUDA / TensorRT19execution providers. One folder per CUDA major version:20 21| Stack | Folder | CUDA | cuDNN | ONNX Runtime EP | When to use |22|---|---|---|---|---|---|23| **cu12** | [`cuda_v12/`](./cuda_v12) | 12.x | 9 | CUDA + TensorRT 10 | sm_75+ GPU on driver **R527+** (CUDA 12) |24| **cu11** | [`cuda_v11/`](./cuda_v11) | 11.x | 8 | CUDA + TensorRT 8.6 | Pascal/Volta (sm_60–sm_70), or any GPU on a driver capped at CUDA 11.x |25 26Pick the stack matching your GPU **and** driver: a newer GPU on an older driver27must still use `cu11` (loading `cu12` binaries fails with28`cudaErrorNoKernelImageForDevice`). The MBFS Sentinel build auto-detects this —29`scripts/build_windows.py` clamps the stack to the lower of the GPU and driver30ceilings (override with `--gpu-stack`).31 32**TensorRT engine-build libraries** live in a *companion* folder,33[`trt_v12/`](./trt_v12) (for the `cu12` stack). These are the TensorRT 1034runtime + parser (`nvinfer*`, `nvonnxparser`) and the per-SM **builder35resources** used when TensorRT compiles an engine. It is **not** a standalone36CUDA stack — pair it with `cuda_v12/` (whose `onnxruntime_providers_tensorrt.dll`37is only the small ONNX Runtime ↔ TensorRT bridge). Unlike the CUDA folders,38`trt_v12/` is fetched **selectively**: a build downloads only the one builder39resource matching the target GPU's compute capability (plus the always-included40PTX fallback), not the whole ~2.8 GiB set. See41**[Contents (`trt_v12/`)](#contents-trt_v12)**.42 43> ⚠️ These are **runtime redistributable libraries**, not source. The CUDA,44> cuDNN, and TensorRT DLLs are © NVIDIA Corporation and remain under NVIDIA's45> licenses — see **[License & redistribution](#license--redistribution)** below.46 47## Contents (`cuda_v12/`)48 49| File | Component | Approx. size |50|---|---|---|51| `onnxruntime.dll` | ONNX Runtime 1.23.2 (combined CUDA + DirectML + TensorRT + CPU build) | 15 MB |52| `onnxruntime_providers_cuda.dll` | ONNX Runtime CUDA execution provider | 361 MB |53| `onnxruntime_providers_tensorrt.dll` | ONNX Runtime TensorRT execution provider | <1 MB |54| `onnxruntime_providers_shared.dll` | ONNX Runtime shared provider interface | <1 MB |55| `cublas64_12.dll`, `cublasLt64_12.dll` | CUDA 12 cuBLAS | 96 / 451 MB |56| `cudart64_12.dll` | CUDA 12 runtime | <1 MB |57| `cufft64_11.dll` | CUDA 12 cuFFT | 279 MB |58| `curand64_10.dll`, `cusparse64_12.dll` | CUDA 12 cuRAND / cuSPARSE | 62 / 264 MB |59| `cudnn64_9.dll` + `cudnn_*64_9.dll` (7 files) | cuDNN 9 | ~1.4 GB |60 61**Total ≈ 2.3 GiB.**62 63## Contents (`cuda_v11/`)64 65| File | Component | Approx. size |66|---|---|---|67| `onnxruntime.dll` | ONNX Runtime (CUDA + TensorRT + CPU build) | 13 MB |68| `onnxruntime_providers_cuda.dll` | ONNX Runtime CUDA execution provider | 223 MB |69| `onnxruntime_providers_tensorrt.dll` | ONNX Runtime TensorRT execution provider | <1 MB |70| `onnxruntime_providers_shared.dll` | ONNX Runtime shared provider interface | <1 MB |71| `cublas64_11.dll`, `cublasLt64_11.dll` | CUDA 11 cuBLAS | 85 / 519 MB |72| `cudart64_110.dll` | CUDA 11 runtime | <1 MB |73| `cufft64_10.dll` | CUDA 11 cuFFT | 267 MB |74| `curand64_10.dll`, `cusparse64_11.dll` | CUDA 11 cuRAND / cuSPARSE | 62 / 265 MB |75| `cudnn64_8.dll` + `cudnn_{adv,cnn,ops}_{infer,train}64_8.dll` (6 files) | cuDNN 8 | ~1.0 GB |76 77**Total ≈ 2.4 GiB.** cuDNN 8 keeps inference and training libraries split78(`*_infer` / `*_train`), unlike cuDNN 9.79 80> All DLLs in a folder come from a single matched build — the ONNX Runtime81> provider DLLs are ABI-locked to that folder's `onnxruntime.dll`, so each set82> must be used together (never mix DLLs across folders or with another ONNX83> Runtime release, or the CUDA EP fails to load).84 85## Contents (`trt_v12/`)86 87TensorRT 10 for the `cu12` stack, split into **core** (always needed) and88**per-SM builder resources** (one per GPU architecture). A build pulls the core89plus **only** the resource matching the target GPU's compute capability, plus90the PTX fallback — typically ≈ 1.0–1.7 GiB instead of the full ≈ 2.8 GiB.91 92**Core — always downloaded:**93 94| File | Component | Approx. size |95|---|---|---|96| `nvinfer_10.dll` | TensorRT 10 core inference runtime + builder | 432 MB |97| `nvinfer_plugin_10.dll` | TensorRT 10 standard plugins | 53 MB |98| `nvonnxparser_10.dll` | ONNX → TensorRT network parser | 3 MB |99| `nvrtc64_120_0.dll` | NVRTC runtime compiler (CUDA 12) | 44 MB |100 101**Per-SM builder resources — download only the one matching your GPU:**102 103| File | Architecture | Example GPUs | Approx. size |104|---|---|---|---|105| `nvinfer_builder_resource_ptx_10.dll` | **PTX JIT fallback — always included** | (any, forward-compat) | 488 MB |106| `nvinfer_builder_resource_sm75_10.dll` | Turing | T4, RTX 20xx | 157 MB |107| `nvinfer_builder_resource_sm80_10.dll` | Ampere | A100, A30 | 256 MB |108| `nvinfer_builder_resource_sm86_10.dll` | Ampere | A40, RTX 30xx | 241 MB |109| `nvinfer_builder_resource_sm89_10.dll` | Ada | L4, L40, RTX 40xx | 254 MB |110| `nvinfer_builder_resource_sm90_10.dll` | Hopper | H100, H200 | 661 MB |111| `nvinfer_builder_resource_sm120_10.dll` | Blackwell | B100/B200, RTX 50xx | 377 MB |112 113**Full set ≈ 2.8 GiB** (4 core + 7 resources). Example single-SM footprint —114Ampere A30 (sm80): core (532 MB) + PTX (488 MB) + `sm80` (256 MB) ≈ **1.2 GiB**.115 116> The builder resources are only needed when TensorRT **compiles** an engine117> (the first run for a given model/GPU). `nvinfer_10.dll` loads the matching118> `nvinfer_builder_resource_sm<cc>_10.dll` from the DLL search path at119> engine-build time; the MBFS Sentinel build stages them under120> `dist/lib/trt_v12/` and adds that directory to the process `PATH`. Once an121> engine cache (`.engine`/`.trt`) exists, the resource for that SM is no longer122> read.123 124## Requirements125 126- **OS:** Windows x64127- **NVIDIA driver:** `cu12` needs **R527+** (CUDA 12.x); `cu11` needs **R452+**128  (CUDA 11.x). The CUDA *Toolkit* does **not** need to be installed — these129  bundles ship the runtime.130- **GPU:** see the support notes below.131 132## GPU support133 134### `cu12`135 136`cuda_v12/onnxruntime_providers_cuda.dll` is compiled with cubins for the137following architectures (no PTX is embedded, so there is **no JIT138forward-compatibility** to newer archs):139 140| Compute capability | Architecture | Example GPUs | Native CUDA |141|---|---|---|---|142| 5.2 | Maxwell | GTX 9xx | ✅ |143| 6.0 | Pascal | Tesla P100 | ✅ |144| 7.0 | Volta | V100 | ✅ |145| 7.5 | Turing | T4, RTX 20xx | ✅ |146| 8.0 | Ampere | A100, A30 | ✅ |147| 8.6 | Ampere | A40, RTX 30xx | ✅ |148| 8.9 | Ada | L4, L40, RTX 40xx | ✅ |149| 9.0a | Hopper | H100, H200 | ✅ |150| 10.x / 12.x | **Blackwell** | B100/B200, RTX 50xx | ❌ not native |151 152For the **TensorRT** path the per-SM builder resources in `trt_v12/` cover153`sm75 / sm80 / sm86 / sm89 / sm90 / sm120`; older archs (sm_52/60/70) fall back154to the PTX builder resource (JIT). Pick the resource by compute capability:155`sm = major*10 + minor` (e.g. 8.0 → `sm80`, 12.0 → `sm120`).156 157### `cu11`158 159The `cu11` stack is the fallback for machines that predate CUDA 12: NVIDIA160**Pascal (sm_60) → Volta (sm_70)**, and any otherwise-cu12-capable GPU running161on a driver that only supports CUDA 11.x. Use it when `nvidia-smi` reports a162CUDA version below 12.0, or for Pascal/Volta silicon. For newer GPUs on a163current driver, prefer `cu12`.164 165On a GPU outside the supported range, or with a too-old driver, the host166application should fall back to DirectML (`DmlExecutionProvider`, requires167`DirectML.dll` — **not** included here) or CPU. These bundles cover the native168CUDA path only.169 170## Usage171 172The MBFS Sentinel build pulls the right stack automatically173(`build_windows.py --gpu-stack {cu12|cu11} --cuda-source hf`), including the174matching TensorRT builder resource (`--tensorrt-source hf`, default). To fetch a175stack manually:176 177```bash178# cu12 into ./dist/lib (lands at ./dist/lib/cuda_v12/)179hf download MBFSAITeam/cuda-runtime-bundle --repo-type dataset \180  --include "cuda_v12/*" --local-dir ./dist/lib181 182# cu11 into ./dist/lib (lands at ./dist/lib/cuda_v11/)183hf download MBFSAITeam/cuda-runtime-bundle --repo-type dataset \184  --include "cuda_v11/*" --local-dir ./dist/lib185```186 187For TensorRT, fetch the **core + PTX + only your GPU's SM** instead of the whole188folder (example for an Ampere A30, sm80):189 190```bash191hf download MBFSAITeam/cuda-runtime-bundle --repo-type dataset \192  --include "trt_v12/nvinfer_10.dll" \193  --include "trt_v12/nvinfer_plugin_10.dll" \194  --include "trt_v12/nvonnxparser_10.dll" \195  --include "trt_v12/nvrtc64_120_0.dll" \196  --include "trt_v12/nvinfer_builder_resource_ptx_10.dll" \197  --include "trt_v12/nvinfer_builder_resource_sm80_10.dll" \198  --local-dir ./dist/lib199```200 201For reproducible builds, pin a commit revision with `--revision <sha>` instead of202`main`.203 204## License & redistribution205 206This repository bundles components under **different licenses**:207 208- **ONNX Runtime** (`onnxruntime*.dll`) — MIT License, © Microsoft.209- **CUDA runtime** (`cudart`, `cublas`, `cublasLt`, `cufft`, `curand`, `cusparse`,210  `nvrtc`) — © NVIDIA Corporation, redistributed under the211  [NVIDIA CUDA Toolkit EULA](https://docs.nvidia.com/cuda/eula/index.html).212- **cuDNN** (`cudnn*.dll`, both v8 and v9) — © NVIDIA Corporation, redistributed213  under the [NVIDIA cuDNN Software License Agreement](https://docs.nvidia.com/deeplearning/cudnn/sla/index.html).214- **TensorRT** (`nvinfer*.dll`, `nvonnxparser*.dll`, and the215  `nvinfer_builder_resource_*` builder resources) — © NVIDIA Corporation,216  redistributed under the217  [NVIDIA TensorRT Software License Agreement](https://docs.nvidia.com/deeplearning/tensorrt/sla/index.html).218 219The NVIDIA libraries are redistributed **unmodified** as runtime dependencies, as220permitted by the above agreements. NVIDIA, CUDA, cuDNN, and TensorRT are trademarks221of NVIDIA Corporation. This repository is not affiliated with or endorsed by NVIDIA.222By using these files you agree to the respective NVIDIA license terms.223