binedge/Qwen3-Embedding-0.6B-FP8

Qwen3-Embedding-0.6B-FP8

FP8-quantized version of `Qwen/Qwen3-Embedding-0.6B`.

This model was quantized with `llm-compressor` using FP8 dynamic activation quantization for the Qwen3 embedding backbone.

Quantization details

• —Base model: Qwen/Qwen3-Embedding-0.6B
• —Quantization tool: llm-compressor
• —Saved format: compressed-tensors
• —Quantization scheme: FP8_DYNAMIC
• —Targets: Linear
• —Ignored modules: none

Quantization recipe

from llmcompressor import oneshot
from llmcompressor.modifiers.quantization import QuantizationModifier

recipe = QuantizationModifier(
    targets="Linear",
    scheme="FP8_DYNAMIC",
)

oneshot(model=model, recipe=recipe)

model.save_pretrained("Qwen3-Embedding-0.6B-FP8")
tokenizer.save_pretrained("Qwen3-Embedding-0.6B-FP8")