Modelpublic

haanjack/EXAONE-4.5-33B-MXFP4

sourceHugging Faceotherupdated 5mo agoView on Hugging Face

0likes13downloads

Model Card

EXAONE-4.5-33B-MXFP4

W4A4 (MXFP4) quantized version of LGAI-EXAONE/EXAONE-4.5-33B, with both weights and activations quantized to OCP MXFP4 format.

EXAONE 4.5 is the first open-weight vision language model developed by LG AI Research. It integrates a dedicated visual encoder into the EXAONE 4.0 framework, expanding the model's capability toward multimodality.

Note: This is an experimental quantized checkpoint. See Limitations for details.

Quantization Details

Property	Value
Base Model	LGAI-EXAONE/EXAONE-4.5-33B (33B params, incl. 1.2B vision encoder)
Precision	W4A4 (MXFP4 Weight + MXFP4 Activation)
Weight Quantization	OCP MXFP4 (E2M1), Static, group_size=32, E8M0 shared scales
Activation Quantization	OCP MXFP4 (E2M1), Dynamic, group_size=32, E8M0 shared scales
Quantization Tool	quanto + AMD Quark 0.11.1 (file-to-file)
Algorithm	RTN (Round-To-Nearest)
Calibration Data	pileval (128 samples)
Original Size	64 GB
Quantized Size	20 GB
Compression Ratio	3.2x

Excluded Layers

The following layers are kept in BF16 (not quantized):

—lm_head — output projection
—Embedding layers (*embed*) — token and vision embeddings
—Normalization layers (*norm*) — RMSNorm layers
—MoE router gates (*.gate) — if present

Model Architecture

Component	Specification
Architecture	Exaone45ForConditionalGeneration (Vision-Language)
Text Backbone	64 decoder layers, hidden_size=5120
Attention	40 heads, 8 KV heads (GQA)
MLP	SwiGLU, intermediate_size=27,392
Vision Encoder	1.2B parameters
Context Length	Up to 262,144 tokens
Reasoning	Supports thinking/non-thinking modes

Limitations

Current

1.Evaluation pending: Accuracy benchmarks (MMLU, KMMLU, etc.) have not been measured yet because exaone4_5 model type is not yet supported in the released version of HuggingFace Transformers. The nuxlear/add-exaone4_5 branch is a work-in-progress.

1.vLLM compatibility unknown: This checkpoint has not been tested with vLLM serving. The Quark MXFP4 weight format (packed uint8 + E8M0 scales) requires vLLM's Quark quantization handler, and EXAONE 4.5 model support in vLLM may also require a custom fork.

1.Vision capabilities not validated: Only the text backbone weights are quantized. The vision encoder weights remain in BF16. Vision-language tasks have not been tested with this quantized checkpoint.

1.No sensitivity analysis applied: Due to the AutoConfig limitation, sensitivity analysis was skipped. All linear layers (except standard exclusions) were quantized uniformly. Accuracy may improve with selective layer exclusion based on sensitivity analysis once transformers support is available.

RTN Algorithm

This checkpoint uses RTN (Round-To-Nearest) quantization, the simplest PTQ method. Advanced algorithms like AWQ or GPTQ may produce better accuracy at the same compression ratio.

Quantization Reproduction

python

from quanto import UnifiedQuantizer, UnifiedConfig

config = UnifiedConfig(
    model_path="LGAI-EXAONE/EXAONE-4.5-33B",
    output_dir="./EXAONE-4.5-33B-MXFP4",
    precision="mxfp4",
    sensitivity_analysis=False,  # AutoConfig fallback used
    skip_evaluation=True,
    trust_remote_code=True,
)
UnifiedQuantizer(config).run()

Credits

—Base Model: LG AI Research — EXAONE 4.5
—Quantization: quanto with AMD Quark
—Hardware: AMD MI355 (gfx950), 288 GB VRAM

License

This model inherits the EXAONE AI Model License Agreement 1.1 - NC from the base model.