Kooten/Lewd-Sydney-20B-4bpw-h8-exl2

Description

Exllama 2 quant of Undi95/Lewd-Sydney-20B

4 BPW, Head bit set to 8

VRAM

My VRAM usage with 20B models are: | Bits per weight | Context | VRAM | |--|--|--| | 6bpw | 4k | 24gb | | 4bpw | 4k | 18gb | | 4bpw | 8k | 24gb | | 3bpw | 4k | 16gb | | 3bpw | 8k | 21gb | I have rounded up, these arent exact numbers, this is also on a windows machine.

Prompt template: Alpaca

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{prompt}

### Response: