dolfsai/Qwen3-Embedding-0.6B-vllm-W8A8

prudant/Qwen3-Embedding-0.6B-W8A8

This is a compressed version of Qwen/Qwen3-Embedding-0.6B using llm-compressor with the following scheme: W8A8

Important: You MUST read the following guide for correct usage of this model here Guide

Model Details

• —Original Model: Qwen/Qwen3-Embedding-0.6B
• —Quantization Method: GPTQ
• —Compression Libraries: llm-compressor
• —Calibration Dataset: ultrachat_200k (1024 samples)
• —Optimized For: Inference with vLLM
• —License: same as original model