bgsach/WizardCoder-Python-13B-V1.0-ct2-int8_float16

This is a quantized version of WizardLM/WizardCoder-Python-13B-V1.0, quantized using ctranslate2 (see inference instructions there).

The license/caveats/intended usage is the same as the original model. The quality of its output may have been negatively affected by the quantization process.

The command run to quantize the model was:

ct2-transformers-converter --model ./models-hf/WizardLM/WizardCoder-Python-13B-V1.0 --quantization int8_float16 --output_dir ./models-ct/WizardLM/WizardCoder-Python-13B-V1.0-ct2-int8_float16

The quantization was run on a 'high-mem', CPU only (8 core, 51GB) colab instance and took approximately 10 minutes.