kaitchup/Olmo-3-7B-Think-w8a8-smoothquant

This is allenai/Olmo-3-7B-Think quantized with LLM Compressor with Smoothquant (W8A8). The model is compatible with vLLM (tested: v0.11.2). Tested with an RTX 4090.

How the models perform (token efficiency, accuracy per domain, ...) and how to use them: Quantizing Olmo 3: Most Efficient and Accurate Formats

• —Developed by: The Kaitchup
• —License: Apache 2.0 license

How to Support My Work

Subscribe to The Kaitchup. This helps me a lot to continue quantizing and evaluating models for free. Or you can "buy me a kofi".