CoolFace
Modelpublic

T3LS/stella-mrl-large-zh-v3.5-1792d-1024-gptq-4bit-exllamav2

sourceHugging Facemitupdated 2y agoView on Hugging Face
1likes5downloads
Model Card

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

Model Details

Model Description

GPTQ-4bit quantization version (use exllamav2) of https://huggingface.co/T3LS/stella-mrl-large-zh-v3.5-1792d-1024

Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

python
model = AutoModelForSequenceClassification.from_pretrained(
    'T3LS/stella-mrl-large-zh-v3.5-1792d-1024-gptq-4bit',
    device_map='cuda' # Exllamav2 backend requires all the modules to be on GPU
)