Modelpublic
ENOSYS/Qwen3-Embedding-0.6B-250-v1-GGUF
1likes959downloads
Model Card
Experimental global target bits‑per‑weight quantization of unsloth/Qwen3-Embedding-0.6B
- —Using non-standard (forked) LLaMA C++ branch for quantization.
- —Using a CLI tool to build KLD evaluation and imatrix calibration datasets for GGUF models, sourced from eaddario/imatrix-calibration.
- —Using dataset sources: tools, texten, textru.
- —Using dataset chunks: 250.
- —Tensors quantinization F16 instead of BF16, Nvidia Pascal architecture friendly like P100.
- —Small set of patches added.
Many thanks to Ed Addario for an impressive job.
Quantization comparison
| BPW | PPL correlation | PPL mean ratio | ΔPPL | Mean KLD | Maximum KLD | 99.9% KLD | Mean Δp | RMS Δp |
|---|---|---|---|---|---|---|---|---|
| 5.00 | 97.51% | 1.229589 ± 0.004913 | 123.695152 ± 3.455547 | 0.254131 ± 0.001247 | 13.281865 | 3.295894 | -0.945 ± 0.030 % | 7.861 ± 0.073 % |
| 5.25 | 97.93% | 1.198584 ± 0.004387 | 106.990729 ± 3.089464 | 0.201051 ± 0.001020 | 16.191372 | 2.680833 | -0.726 ± 0.027 % | 7.070 ± 0.070 % |
| 5.30 | 98.05% | 1.199072 ± 0.004269 | 107.253986 ± 3.067627 | 0.181907 ± 0.000926 | 11.642469 | 2.371547 | -0.680 ± 0.026 % | 6.811 ± 0.067 % |
| 5.50 | 98.42% | 1.143577 ± 0.003665 | 77.354913 ± 2.488164 | 0.132384 ± 0.000693 | 9.056372 | 1.854777 | -0.483 ± 0.022 % | 5.860 ± 0.061 % |
| 5.75 | 98.67% | 1.107662 ± 0.003264 | 58.005137 ± 2.117964 | 0.097616 ± 0.000599 | 11.806184 | 1.765460 | -0.368 ± 0.019 % | 5.023 ± 0.062 % |
| 5.80 | 98.72% | 1.117510 ± 0.003238 | 63.310457 ± 2.168267 | 0.092513 ± 0.000578 | 10.540298 | 1.608747 | -0.386 ± 0.019 % | 4.906 ± 0.059 % |
| 6.00 | 98.71% | 1.125959 ± 0.003273 | 67.862792 ± 2.234984 | 0.092148 ± 0.000585 | 12.128985 | 1.559407 | -0.435 ± 0.019 % | 4.931 ± 0.061 % |
| 6.25 | 98.89% | 1.082024 ± 0.002912 | 44.191950 ± 1.831029 | 0.067634 ± 0.000431 | 7.849868 | 1.006397 | -0.264 ± 0.016 % | 4.210 ± 0.055 % |
| 6.30 | 98.94% | 1.089430 ± 0.002886 | 48.181944 ± 1.865568 | 0.062879 ± 0.000377 | 7.467944 | 0.950639 | -0.262 ± 0.016 % | 4.112 ± 0.055 % |
| 6.50 | 99.07% | 1.110195 ± 0.002773 | 59.369600 ± 1.962020 | 0.046909 ± 0.000273 | 5.759321 | 0.702734 | -0.293 ± 0.013 % | 3.510 ± 0.044 % |
| 6.75 | 99.22% | 1.071155 ± 0.002479 | 38.336145 ± 1.614146 | 0.027021 ± 0.000217 | 6.292867 | 0.423712 | -0.069 ± 0.011 % | 2.756 ± 0.051 % |
| 6.80 | 99.23% | 1.079566 ± 0.002483 | 42.867759 ± 1.667153 | 0.026098 ± 0.000205 | 6.737493 | 0.397282 | -0.098 ± 0.010 % | 2.714 ± 0.051 % |
| 7.00 | 99.24% | 1.083987 ± 0.002490 | 45.249386 ± 1.706124 | 0.023685 ± 0.000190 | 6.608492 | 0.377540 | -0.084 ± 0.010 % | 2.559 ± 0.049 % |
| 7.25 | 99.27% | 1.088304 ± 0.002447 | 47.575202 ± 1.717173 | 0.019738 ± 0.000130 | 2.865925 | 0.332578 | -0.090 ± 0.009 % | 2.290 ± 0.037 % |
| 7.30 | 99.28% | 1.084746 ± 0.002430 | 45.658594 ± 1.686292 | 0.019120 ± 0.000148 | 4.343514 | 0.295245 | -0.092 ± 0.009 % | 2.291 ± 0.048 % |
| 7.50 | 99.29% | 1.085544 ± 0.002411 | 46.088344 ± 1.685529 | 0.017551 ± 0.000142 | 4.905227 | 0.298486 | -0.078 ± 0.008 % | 2.215 ± 0.050 % |
| 7.75 | 99.32% | 1.091225 ± 0.002385 | 49.149079 ± 1.714694 | 0.014173 ± 0.000128 | 4.346512 | 0.237379 | -0.101 ± 0.008 % | 2.013 ± 0.052 % |
| 7.80 | 99.29% | 1.083209 ± 0.002410 | 44.830380 ± 1.669455 | 0.016992 ± 0.000133 | 3.100509 | 0.285571 | -0.077 ± 0.008 % | 2.162 ± 0.041 % |
| 8.00 | 99.31% | 1.080260 ± 0.002373 | 43.241571 ± 1.636024 | 0.015367 ± 0.000130 | 4.035001 | 0.269038 | -0.060 ± 0.008 % | 2.071 ± 0.049 % |
| 8.25 | 99.33% | 1.088309 ± 0.002352 | 47.578112 ± 1.683232 | 0.012024 ± 0.000105 | 4.370751 | 0.192501 | -0.089 ± 0.007 % | 1.860 ± 0.044 % |
| 8.30 | 99.34% | 1.081929 ± 0.002328 | 44.140973 ± 1.628892 | 0.011498 ± 0.000084 | 1.861967 | 0.182563 | -0.075 ± 0.007 % | 1.784 ± 0.037 % |
| 8.50 | 99.36% | 1.078487 ± 0.002286 | 42.286422 ± 1.587902 | 0.009095 ± 0.000081 | 2.475720 | 0.147575 | -0.064 ± 0.006 % | 1.635 ± 0.048 % |
| 8.75 | 99.37% | 1.078905 ± 0.002274 | 42.511791 ± 1.587059 | 0.007852 ± 0.000073 | 2.163608 | 0.118868 | -0.054 ± 0.006 % | 1.535 ± 0.049 % |
| 8.80 | 99.37% | 1.078538 ± 0.002273 | 42.314029 ± 1.583733 | 0.007742 ± 0.000079 | 2.609957 | 0.127498 | -0.053 ± 0.006 % | 1.533 ± 0.053 % |
| 9.00 | 99.37% | 1.077841 ± 0.002265 | 41.938514 ± 1.575722 | 0.007380 ± 0.000073 | 2.547039 | 0.125235 | -0.050 ± 0.006 % | 1.499 ± 0.053 % |
| 9.25 | 99.37% | 1.075351 ± 0.002256 | 40.596804 ± 1.555539 | 0.006905 ± 0.000066 | 2.041534 | 0.103068 | -0.042 ± 0.006 % | 1.448 ± 0.044 % |
| 9.30 | 99.37% | 1.071370 ± 0.002244 | 38.451639 ± 1.524417 | 0.006834 ± 0.000062 | 1.959199 | 0.105753 | -0.031 ± 0.006 % | 1.444 ± 0.040 % |
| 9.50 | 99.38% | 1.073834 ± 0.002244 | 39.779313 ± 1.540846 | 0.006470 ± 0.000062 | 1.991340 | 0.098428 | -0.041 ± 0.005 % | 1.400 ± 0.041 % |
| 9.75 | 99.38% | 1.074517 ± 0.002240 | 40.147460 ± 1.543907 | 0.006206 ± 0.000068 | 2.638701 | 0.101805 | -0.038 ± 0.005 % | 1.388 ± 0.053 % |
| 9.80 | 99.38% | 1.076663 ± 0.002244 | 41.303655 ± 1.559774 | 0.006146 ± 0.000069 | 2.405736 | 0.094807 | -0.044 ± 0.005 % | 1.401 ± 0.057 % |
| 10.00 | 99.38% | 1.076016 ± 0.002240 | 40.955259 ± 1.553864 | 0.005796 ± 0.000055 | 2.059185 | 0.095681 | -0.049 ± 0.005 % | 1.313 ± 0.039 % |
| 10.25 | 99.39% | 1.074654 ± 0.002230 | 40.220957 ± 1.540922 | 0.005526 ± 0.000061 | 2.114943 | 0.099634 | -0.043 ± 0.005 % | 1.315 ± 0.050 % |
| 10.30 | 99.39% | 1.072069 ± 0.002222 | 38.828737 ± 1.519978 | 0.005418 ± 0.000059 | 2.062696 | 0.094137 | -0.038 ± 0.005 % | 1.288 ± 0.047 % |
| 10.50 | 99.39% | 1.070241 ± 0.002212 | 37.843815 ± 1.503987 | 0.005140 ± 0.000048 | 1.621503 | 0.087822 | -0.032 ± 0.005 % | 1.218 ± 0.028 % |
| 10.75 | 99.39% | 1.066075 ± 0.002195 | 35.599290 ± 1.470596 | 0.004415 ± 0.000060 | 2.666409 | 0.074945 | -0.011 ± 0.004 % | 1.152 ± 0.049 % |
| 10.80 | 99.40% | 1.066699 ± 0.002195 | 35.935359 ± 1.473675 | 0.004566 ± 0.000051 | 2.669465 | 0.074244 | -0.025 ± 0.004 % | 1.109 ± 0.017 % |
| 11.00 | 99.40% | 1.071004 ± 0.002207 | 38.254894 ± 1.509298 | 0.004029 ± 0.000035 | 1.280535 | 0.060520 | -0.019 ± 0.004 % | 1.112 ± 0.045 % |
| 11.25 | 99.40% | 1.071390 ± 0.002201 | 38.462580 ± 1.508764 | 0.003687 ± 0.000024 | 0.614354 | 0.054238 | -0.025 ± 0.004 % | 0.995 ± 0.011 % |
| 11.30 | 99.40% | 1.071772 ± 0.002202 | 38.668624 ± 1.511744 | 0.003665 ± 0.000024 | 0.613789 | 0.055209 | -0.023 ± 0.004 % | 0.984 ± 0.011 % |
| 11.50 | 99.40% | 1.072891 ± 0.002200 | 39.271504 ± 1.518324 | 0.003498 ± 0.000022 | 0.426572 | 0.052573 | -0.031 ± 0.004 % | 0.977 ± 0.013 % |
| 11.75 | 99.41% | 1.067707 ± 0.002184 | 36.478332 ± 1.478998 | 0.002967 ± 0.000023 | 0.803354 | 0.045490 | -0.001 ± 0.003 % | 0.895 ± 0.013 % |
| 11.80 | 99.41% | 1.065799 ± 0.002180 | 35.450545 ± 1.465811 | 0.002931 ± 0.000021 | 0.697018 | 0.045669 | 0.006 ± 0.003 % | 0.899 ± 0.018 % |