Modelpublic

nm-testing/tinyllama-one-shot-static-quant-test-compressed

sourceHugging Faceupdated 2y agoView on Hugging Face

0likes47downloads

recipe.yaml11 linesDownload Raw Back to root

1test_stage:2  quant_modifiers:3    vLLMQuantizationModifier:4      ignore: [lm_head]5      config_groups:6        group_0:7          weights: {num_bits: 8, type: int, symmetric: true, strategy: tensor}8          input_activations: {num_bits: 8, type: int, symmetric: true, strategy: tensor}9          targets: [Linear]10    SparseGPTModifier: {sparsity: 0.0, quantize: true, sequential_update: false}11