nzgnzg73/llama_cpp_WebUI

1---2license: apache-2.03tags:4- Image-Text-to-Text Models5- Audio-Text-to-Text6- text-to-Text7- llama_cpp8- any to any9- Multimodal AI10- video-Text-to-Text11- Llama Model Switcher12---13nzgnzg7314 15llama_cpp_WebUI16 17Github18https://github.com/nzgnzg73/llama_cpp_WebUI19 20## Want to talk or ask something?21Just click the YouTube link below! You'll find my 📧 email there and can message me easily. 👇22 23🎥 YouTube Channel: @nzg7324🔗 https://youtube.com/@NZG7325 26 27## Contact Email 👇👇👀28E-mail:-29nzgnzg73@gmail.com30 31 32llama Cpp (cp\gpu)33Old Version llama cpp 👇👇👇👇34llama-b7200-bin-win-cpu-x64.zip35 36 37NEW Update Version llama cpp 👇👇👇👇38 39llama-b7541-bin-win-cpu-x6440 41llama-b7243-bin-win-cuda-12.4-x6442 43 44 45![Gemini_Generated_Image_rx1z0erx1z0erx1z](https://cdn-uploads.huggingface.co/production/uploads/68edac11571026107ce50f6c/7Cjp8zMEtdftA0lfJzKZL.png)46 47 48## Llama Model Switcher49 50CMD51model_switcher.py52 53pip install flask54 55pip install flask psutil GPUtil56 57![1](https://cdn-uploads.huggingface.co/production/uploads/68edac11571026107ce50f6c/PUekE98unK4E-wCue-Dle.png)58 59 60 61![2](https://cdn-uploads.huggingface.co/production/uploads/68edac11571026107ce50f6c/CZvP7V61-SE4o4_Pu38_s.png)62 63## Image-Text-to-Text Models64 65## Gemma-366 67CPU.  RAM 20GB68OR69GPU.  4 VRAM70 71 72![1](https://cdn-uploads.huggingface.co/production/uploads/68edac11571026107ce50f6c/qoYFWiyKbxfec608p3AbF.png)73 741. gemma-3-12b-it-Q4_K_S.gguf752. mmproj-model-f16-12B.gguf76 77 78## -Text-to-Text Models79## GPT OSS 2080 81 82 83 84![2](https://cdn-uploads.huggingface.co/production/uploads/68edac11571026107ce50f6c/lYXBozCQV-SmpXDk4ca9I.png)85 86 87 88 89 90![Picsart_25-11-30_03-51-56-362](https://cdn-uploads.huggingface.co/production/uploads/68edac11571026107ce50f6c/II21vWhc6UJwMwAqw1qaQ.png)91 92 93 94 95## Qwen396 97 98CPU.  RAM 25GB99OR100GPU.  4 VRAM101 102 1031. Qwen3-VL-2B-Instruct-Q8_0.gguf1042. mmproj-Qwen3-VL-2B-Instruct-Q8_0.gguf105 106 107 108![2](https://cdn-uploads.huggingface.co/production/uploads/68edac11571026107ce50f6c/jmba7H290hQTPhBVL-v5Q.png)109 110 111## Qwen2.5-Omni112 113 114CPU.  RAM 40GB115OR116GPU.  8 VRAM117 1181. Qwen2.5-Omni-7B-BF16.gguf  1192. mmproj-F16.gguf           2GB120 121 122 123![4](https://cdn-uploads.huggingface.co/production/uploads/68edac11571026107ce50f6c/v0ACCIgBmFUmcMH5rTjj_.png)124 125 126 127 128 129 130 131 132 133## Audio-Text-to-Text134 135 136## Llama-3.2137 138 139CPU.  RAM 10GB140 141 142 143 144![3](https://cdn-uploads.huggingface.co/production/uploads/68edac11571026107ce50f6c/VJ9-OMFQYX8x-4HhhtknP.png)145 146 1471. Llama-3.2-1B-Instruct-Q4_K_M.gguf1482. Llama-3.2-1B-Instruct-Q8_0.gguf1493. mmproj-ultravox-v0_5-llama-3_2-1b-f16.gguf150 151 152 153## run.bat154 155 156 157 158Local Server159 160llama-server.exe --n-gpu-layers 2 --ctx-size 111192 -m ".\models\mistralai\mistralai_Voxtral-Mini-3B-2507-Q8_0.gguf" --mmproj ".\models\mistralai\mmproj-mistralai_Voxtral-Mini-3B-2507-bf16.gguf" --host 0.0.0.0 --port 8005161 162 163public URL164 165llama-server --n-gpu-layers 15 --ctx-size 8192 -m models/ollma/Llama-3.2-1B-Instruct-Q8_0.gguf --mmproj models/ollma/mmproj-ultravox-v0_5-llama-3_2-1b-f16.gguf --host 127.0.0.1 --port 8083