forestcalled/text-generation-webui
0
1Docker Compose is a way of installing and launching the web UI in an isolated Ubuntu image using only a few commands.2 3## Installing Docker Compose4 5In order to create the image as described in the main README, you must have Docker Compose installed (2.17 or higher is recommended):6 7```8~$ docker compose version9Docker Compose version v2.21.010```11 12The installation instructions for various Linux distributions can be found here:13 14https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository15 16## Launching the image17 18Use these commands to launch the image:19 20```21cd text-generation-webui22ln -s docker/{nvidia/Dockerfile,docker-compose.yml,.dockerignore} .23cp docker/.env.example .env24# Edit .env and set TORCH_CUDA_ARCH_LIST based on your GPU model25docker compose up --build26```27 28## More detailed installation instructions29 30* [Docker Compose installation instructions](#docker-compose-installation-instructions)31* [Repository with additional Docker files](#dedicated-docker-repository)32 33By [@loeken](https://github.com/loeken).34 35- [Ubuntu 22.04](#ubuntu-2204)36 - [0. youtube video](#0-youtube-video)37 - [1. update the drivers](#1-update-the-drivers)38 - [2. reboot](#2-reboot)39 - [3. install docker](#3-install-docker)40 - [4. docker \& container toolkit](#4-docker--container-toolkit)41 - [5. clone the repo](#5-clone-the-repo)42 - [6. prepare models](#6-prepare-models)43 - [7. prepare .env file](#7-prepare-env-file)44 - [8. startup docker container](#8-startup-docker-container)45- [Manjaro](#manjaro)46 - [update the drivers](#update-the-drivers)47 - [reboot](#reboot)48 - [docker \& container toolkit](#docker--container-toolkit)49 - [continue with ubuntu task](#continue-with-ubuntu-task)50- [Windows](#windows)51 - [0. youtube video](#0-youtube-video-1)52 - [1. choco package manager](#1-choco-package-manager)53 - [2. install drivers/dependencies](#2-install-driversdependencies)54 - [3. install wsl](#3-install-wsl)55 - [4. reboot](#4-reboot)56 - [5. git clone \&\& startup](#5-git-clone--startup)57 - [6. prepare models](#6-prepare-models-1)58 - [7. startup](#7-startup)59- [notes](#notes)60 61### Ubuntu 22.0462 63#### 0. youtube video64A video walking you through the setup can be found here:65 66[](https://www.youtube.com/watch?v=ELkKWYh8qOk)67 68 69#### 1. update the drivers70in the the “software updater” update drivers to the last version of the prop driver.71 72#### 2. reboot73to switch using to new driver74 75#### 3. install docker76```bash77sudo apt update78sudo apt-get install curl79sudo mkdir -m 0755 -p /etc/apt/keyrings80curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg81echo \82 "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \83 "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \84 sudo tee /etc/apt/sources.list.d/docker.list > /dev/null85sudo apt update86sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin docker-compose -y87sudo usermod -aG docker $USER88newgrp docker89```90 91#### 4. docker & container toolkit92```bash93curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg94echo "deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://nvidia.github.io/libnvidia-container/stable/ubuntu22.04/amd64 /" | \95sudo tee /etc/apt/sources.list.d/nvidia.list > /dev/null 96sudo apt update97sudo apt install nvidia-docker2 nvidia-container-runtime -y98sudo systemctl restart docker99```100 101#### 5. clone the repo102```103git clone https://github.com/oobabooga/text-generation-webui104cd text-generation-webui105```106 107#### 6. prepare models108download and place the models inside the models folder. tested with:109 1104bit111https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483891617112https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483941105113 1148bit:115https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1484235789116 117#### 7. prepare .env file118edit .env values to your needs.119```bash120cp .env.example .env121nano .env122```123 124#### 8. startup docker container125```bash126docker compose up --build127```128 129### Manjaro130manjaro/arch is similar to ubuntu just the dependency installation is more convenient131 132#### update the drivers133```bash134sudo mhwd -a pci nonfree 0300135```136#### reboot137```bash138reboot139```140#### docker & container toolkit141```bash142yay -S docker docker-compose buildkit gcc nvidia-docker143sudo usermod -aG docker $USER144newgrp docker145sudo systemctl restart docker # required by nvidia-container-runtime146```147 148#### continue with ubuntu task149continue at [5. clone the repo](#5-clone-the-repo)150 151### Windows152#### 0. youtube video153A video walking you through the setup can be found here:154[](https://www.youtube.com/watch?v=ejH4w5b5kFQ)155 156#### 1. choco package manager157install package manager (https://chocolatey.org/ )158```159Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))160```161 162#### 2. install drivers/dependencies163```164choco install nvidia-display-driver cuda git docker-desktop165```166 167#### 3. install wsl168wsl --install169 170#### 4. reboot171after reboot enter username/password in wsl172 173#### 5. git clone && startup174clone the repo and edit .env values to your needs.175```176cd Desktop177git clone https://github.com/oobabooga/text-generation-webui178cd text-generation-webui179COPY .env.example .env180notepad .env181```182 183#### 6. prepare models184download and place the models inside the models folder. tested with:185 1864bit https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483891617 https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483941105187 1888bit: https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1484235789189 190#### 7. startup191```192docker compose up193```194 195### notes196 197on older ubuntus you can manually install the docker compose plugin like this:198```199DOCKER_CONFIG=${DOCKER_CONFIG:-$HOME/.docker}200mkdir -p $DOCKER_CONFIG/cli-plugins201curl -SL https://github.com/docker/compose/releases/download/v2.17.2/docker-compose-linux-x86_64 -o $DOCKER_CONFIG/cli-plugins/docker-compose202chmod +x $DOCKER_CONFIG/cli-plugins/docker-compose203export PATH="$HOME/.docker/cli-plugins:$PATH"204```205 206## Dedicated docker repository207 208An external repository maintains a docker wrapper for this project as well as several pre-configured 'one-click' `docker compose` variants (e.g., updated branches of GPTQ). It can be found at: [Atinoda/text-generation-webui-docker](https://github.com/Atinoda/text-generation-webui-docker).209