binedge/dots.mocr-FP8

dots.mocr-FP8

FP8-quantized version of `rednote-hilab/dots.mocr`.

This model was quantized with `llm-compressor` using FP8 dynamic activation quantization for the text backbone. The custom vision tower was intentionally excluded from quantization and kept in BF16.

Quantization details

• —Base model: rednote-hilab/dots.mocr
• —Quantization tool: llm-compressor
• —Saved format: compressed-tensors
• —Quantization scheme: FP8_DYNAMIC
• —Targets: Linear
• —Ignored modules:
• —lm_head
• —.*vision_tower.*

Quantization recipe

from llmcompressor import oneshot
from llmcompressor.modifiers.quantization import QuantizationModifier

recipe = QuantizationModifier(
    targets="Linear",
    scheme="FP8_DYNAMIC",
    ignore=[
        "lm_head",
        "re:.*vision_tower.*",
    ],
)

oneshot(model=model, recipe=recipe)

model.save_pretrained("binedge/dots.mocr-FP8", save_compressed=True)
processor.save_pretrained("binedge/dots.mocr-FP8")