Modelpublic

Edge-Quant/EXAONE-4.0-1.2B-Q4_K_M-GGUF

sourceHugging Faceotherupdated 10mo agoView on Hugging Face

0likes28downloads

Model Card

Edge-Quant/EXAONE-4.0-1.2B-Q4KM-GGUF

This model was converted to GGUF format from `LGAI-EXAONE/EXAONE-4.0-1.2B` using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model.

benchmark 1.2B Reasoning Mode

<table> <tr> <th> </th> <th>EXAONE 4.0 1.2B </th> <th>EXAONE Deep 2.4B</th> <th>Qwen 3 0.6B </th> <th>Qwen 3 1.7B </th> <th>SmolLM 3 3B </th> </tr> <tr> <td align="center">Model Size</td> <td align="center">1.28B</td> <td align="center">2.41B</td> <td align="center">596M</td> <td align="center">1.72B</td> <td align="center">3.08B</td> </tr> <tr> <td align="center">Hybrid Reasoning</td> <td align="center">✅</td> <td align="center"> </td> <td align="center">✅</td> <td align="center">✅</td> <td align="center">✅</td> </tr> <tr> <td align="center" colspan='6'>World Knowledge</td> </tr> <tr> <td >MMLU-Redux</td> <td align="center">71.5</td> <td align="center">68.9</td> <td align="center">55.6</td> <td align="center">73.9</td> <td align="center">74.8</td> </tr> <tr> <td >MMLU-Pro</td> <td align="center">59.3</td> <td align="center">56.4</td> <td align="center">38.3</td> <td align="center">57.7</td> <td align="center">57.8</td> </tr> <tr> <td >GPQA-Diamond</td> <td align="center">52.0</td> <td align="center">54.3</td> <td align="center">27.9</td> <td align="center">40.1</td> <td align="center">41.7</td> </tr> <tr> <td align="center" colspan='6'>Math/Coding</td> </tr> <tr> <td >AIME 2025</td> <td align="center">45.2</td> <td align="center">47.9</td> <td align="center">15.1</td> <td align="center">36.8</td> <td align="center">36.7</td> </tr> <tr> <td >HMMT Feb 2025</td> <td align="center">34.0</td> <td align="center">27.3</td> <td align="center">7.0</td> <td align="center">21.8</td> <td align="center">26.0</td> </tr> <tr> <td >LiveCodeBench v5</td> <td align="center">44.6</td> <td align="center">47.2</td> <td align="center">12.3</td> <td align="center">33.2</td> <td align="center">27.6</td> </tr> <tr> <td >LiveCodeBench v6</td> <td align="center">45.3</td> <td align="center">43.1</td> <td align="center">16.4</td> <td align="center">29.9</td> <td align="center">29.1</td> </tr> <tr> <td align="center" colspan='6'>Instruction Following</td> </tr> <tr> <td >IFEval</td> <td align="center">67.8</td> <td align="center">71.0</td> <td align="center">59.2</td> <td align="center">72.5</td> <td align="center">71.2</td> </tr> <tr> <td >Multi-IF (EN)</td> <td align="center">53.9</td> <td align="center">54.5</td> <td align="center">37.5</td> <td align="center">53.5</td> <td align="center">47.5</td> </tr> <tr> <td align="center" colspan='6'>Agentic Tool Use</td> </tr> <tr> <td >BFCL-v3</td> <td align="center">52.9</td> <td align="center">N/A</td> <td align="center">46.4</td> <td align="center">56.6</td> <td align="center">37.1</td> </tr> <tr> <td >Tau-Bench (Airline)</td> <td align="center">20.5</td> <td align="center">N/A</td> <td align="center">22.0</td> <td align="center">31.0</td> <td align="center">37.0</td> </tr> <tr> <td >Tau-Bench (Retail)</td> <td align="center">28.1</td> <td align="center">N/A</td> <td align="center">3.3</td> <td align="center">6.5</td> <td align="center">5.4</td> </tr> <tr> <td align="center" colspan='6'>Multilinguality</td> </tr> <tr> <td >KMMLU-Pro</td> <td align="center">42.7</td> <td align="center">24.6</td> <td align="center">21.6</td> <td align="center">38.3</td> <td align="center">30.5</td> </tr> <tr> <td >KMMLU-Redux</td> <td align="center">46.9</td> <td align="center">25.0</td> <td align="center">24.5</td> <td align="center">38.0</td> <td align="center">33.7</td> </tr> <tr> <td >KSM</td> <td align="center">60.6</td> <td align="center">60.9</td> <td align="center">22.8</td> <td align="center">52.9</td> <td align="center">49.7</td> </tr> <tr> <td >MMMLU (ES)</td> <td align="center">62.4</td> <td align="center">51.4</td> <td align="center">48.8</td> <td align="center">64.5</td> <td align="center">64.7</td> </tr> <tr> <td >MATH500 (ES)</td> <td align="center">88.8</td> <td align="center">84.5</td> <td align="center">70.6</td> <td align="center">87.9</td> <td align="center">87.5 </td> </tr> </table>

1.2B Non-Reasoning Mode

<table> <tr> <th> </th> <th>EXAONE 4.0 1.2B </th> <th>Qwen 3 0.6B </th> <th>Gemma 3 1B</th> <th>Qwen 3 1.7B </th> <th>SmolLM 3 3B </th> </tr> <tr> <td align="center">Model Size</td> <td align="center">1.28B</td> <td align="center">596M</td> <td align="center">1.00B</td> <td align="center">1.72B</td> <td align="center">3.08B</td> </tr> <tr> <td align="center">Hybrid Reasoning</td> <td align="center">✅</td> <td align="center">✅</td> <td align="center"> </td> <td align="center">✅</td> <td align="center">✅</td> </tr> <tr> <td align="center" colspan='6'>World Knowledge</td> </tr> <tr> <td >MMLU-Redux</td> <td align="center">66.9</td> <td align="center">44.6</td> <td align="center">40.9</td> <td align="center">63.4</td> <td align="center">65.0</td> </tr> <tr> <td >MMLU-Pro</td> <td align="center">52.0</td> <td align="center">26.6</td> <td align="center">14.7</td> <td align="center">43.7</td> <td align="center">43.6</td> </tr> <tr> <td >GPQA-Diamond</td> <td align="center">40.1</td> <td align="center">22.9</td> <td align="center">19.2</td> <td align="center">28.6</td> <td align="center">35.7</td> </tr> <tr> <td align="center" colspan='6'>Math/Coding</td> </tr> <tr> <td >AIME 2025</td> <td align="center">23.5</td> <td align="center">2.6</td> <td align="center">2.1</td> <td align="center">9.8</td> <td align="center">9.3</td> </tr> <tr> <td >HMMT Feb 2025</td> <td align="center">13.0</td> <td align="center">1.0</td> <td align="center">1.5</td> <td align="center">5.1</td> <td align="center">4.7</td> </tr> <tr> <td >LiveCodeBench v5</td> <td align="center">26.4</td> <td align="center">3.6</td> <td align="center">1.8</td> <td align="center">11.6</td> <td align="center">11.4</td> </tr> <tr> <td >LiveCodeBench v6</td> <td align="center">30.1</td> <td align="center">6.9</td> <td align="center">2.3</td> <td align="center">16.6</td> <td align="center">20.6</td> </tr> <tr> <td align="center" colspan='6'>Instruction Following</td> </tr> <tr> <td >IFEval</td> <td align="center">74.7</td> <td align="center">54.5</td> <td align="center">80.2</td> <td align="center">68.2</td> <td align="center">76.7</td> </tr> <tr> <td >Multi-IF (EN)</td> <td align="center">62.1</td> <td align="center">37.5</td> <td align="center">32.5</td> <td align="center">51.0</td> <td align="center">51.9</td> </tr> <tr> <td align="center" colspan='6'>Long Context</td> </tr> <tr> <td >HELMET</td> <td align="center">41.2</td> <td align="center">21.1</td> <td align="center">N/A</td> <td align="center">33.8</td> <td align="center">38.6</td> </tr> <tr> <td >RULER</td> <td align="center">77.4</td> <td align="center">55.1</td> <td align="center">N/A</td> <td align="center">65.9</td> <td align="center">66.3</td> </tr> <tr> <td >LongBench v1</td> <td align="center">36.9</td> <td align="center">32.4</td> <td align="center">N/A</td> <td align="center">41.9</td> <td align="center">39.9</td> </tr> <tr> <td align="center" colspan='6'>Agentic Tool Use</td> </tr> <tr> <td >BFCL-v3</td> <td align="center">55.7</td> <td align="center">44.1</td> <td align="center">N/A</td> <td align="center">52.2</td> <td align="center">47.3</td> </tr> <tr> <td >Tau-Bench (Airline)</td> <td align="center">10.0</td> <td align="center">31.5</td> <td align="center">N/A</td> <td align="center">13.5</td> <td align="center">38.0</td> </tr> <tr> <td >Tau-Bench (Retail)</td> <td align="center">21.7</td> <td align="center">5.7</td> <td align="center">N/A</td> <td align="center">4.6</td> <td align="center">6.7</td> </tr> <tr> <td align="center" colspan='6'>Multilinguality</td> </tr> <tr> <td >KMMLU-Pro</td> <td align="center">37.5</td> <td align="center">24.6</td> <td align="center">9.7</td> <td align="center">29.5</td> <td align="center">27.6</td> </tr> <tr> <td >KMMLU-Redux</td> <td align="center">40.4</td> <td align="center">22.8</td> <td align="center">19.4</td> <td align="center">29.8</td> <td align="center">26.4</td> </tr> <tr> <td >KSM</td> <td align="center">26.3</td> <td align="center">0.1</td> <td align="center">22.8</td> <td align="center">16.3</td> <td align="center">16.1</td> </tr> <tr> <td >Ko-LongBench</td> <td align="center">69.8</td> <td align="center">16.4</td> <td align="center">N/A</td> <td align="center">57.1</td> <td align="center">15.7</td> </tr> <tr> <td >MMMLU (ES)</td> <td align="center">54.6</td> <td align="center">39.5</td> <td align="center">35.9</td> <td align="center">54.3</td> <td align="center">55.1</td> </tr> <tr> <td >MATH500 (ES)</td> <td align="center">71.2</td> <td align="center">38.5</td> <td align="center">41.2</td> <td align="center">66.0</td> <td align="center">62.4</td> </tr> <tr> <td >WMT24++ (ES)</td> <td align="center">65.9</td> <td align="center">58.2</td> <td align="center">76.9</td> <td align="center">76.7</td> <td align="center">84.0 </td> </tr> </table>

Use with llama.cpp

Install llama.cpp through brew (works on Mac and Linux)

bash

brew install llama.cpp

Invoke the llama.cpp server or the CLI.

CLI:

bash

llama-cli --hf-repo Edge-Quant/EXAONE-4.0-1.2B-Q4_K_M-GGUF --hf-file exaone-4.0-1.2b-q4_k_m.gguf -p "The meaning to life and the universe is"

Server:

bash

llama-server --hf-repo Edge-Quant/EXAONE-4.0-1.2B-Q4_K_M-GGUF --hf-file exaone-4.0-1.2b-q4_k_m.gguf -c 2048

Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.

git clone https://github.com/ggerganov/llama.cpp

Step 2: Move into the llama.cpp folder and build it with LLAMA_CURL=1 flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).

cd llama.cpp && LLAMA_CURL=1 make

Step 3: Run inference through the main binary.

./llama-cli --hf-repo Edge-Quant/EXAONE-4.0-1.2B-Q4_K_M-GGUF --hf-file exaone-4.0-1.2b-q4_k_m.gguf -p "The meaning to life and the universe is"

./llama-server --hf-repo Edge-Quant/EXAONE-4.0-1.2B-Q4_K_M-GGUF --hf-file exaone-4.0-1.2b-q4_k_m.gguf -c 2048