WhirlwindAI/Arithmetic-SLM

Scores

<div align="center">

<table> <tr> <th align="center">Model</th> <th align="center">Parameters</th> <th align="center">Overall Score</th> </tr> <tr> <td align="center"><code>Qwen/Qwen2.5-Math-1.5B</code></td> <td align="center">1.54B</td> <td align="center"><strong>82.08%</strong></td> </tr> <tr> <td align="center"><code>WhirlwindAI/Arithmetic-SLM</code></td> <td align="center">31.70M</td> <td align="center"><strong>78.60%</strong></td> </tr> <tr> <td align="center"><code>Qwen/Qwen2.5-3B</code></td> <td align="center">3.09B</td> <td align="center">78.44%</td> </tr> <tr> <td align="center"><code>Qwen/Qwen2.5-1.5B</code></td> <td align="center">1.54B</td> <td align="center">77.72%</td> </tr> <tr> <td align="center"><code>Qwen/Qwen2.5-Coder-1.5B</code></td> <td align="center">1.54B</td> <td align="center">74.88%</td> </tr> <tr> <td align="center"><code>HuggingFaceTB/SmolLM2-1.7B</code></td> <td align="center">1.71B</td> <td align="center">66.12%</td> </tr> <tr> <td align="center"><code>Qwen/Qwen2.5-0.5B</code></td> <td align="center">494M</td> <td align="center">63.04%</td> </tr> <tr> <td align="center"><code>facebook/MobileLLM-R1-140M-base</code></td> <td align="center">140M</td> <td align="center">53.88%</td> </tr> <tr> <td align="center"><code>SupraLabs/Supra-50M-Base</code></td> <td align="center">52M</td> <td align="center">27.12%</td> </tr> </table>

</div>

Arithmetic-SLM

Arithmetic-SLM is a small language model specialized for arithmetic continuation. It is designed to be highly efficient on numerical operations with mostly two-digit numbers in patterns such as:

a op b op c op d

where:

op = +, -, *, /

The goal is not to make a general chatbot. The goal is to train a compact model that can learn arithmetic patterns, operator priority, parentheses, and numerical continuation with very few parameters.

Calculation Patterns

1. Single operation

59 + 45 = 104
26 - 2 = 24
12 * 7 = 84
84 / 12 = 7

2. Two operations without parentheses

16 + 4 * 3 = 28
95 - 8 * 0 = 95
84 / 12 - 3 = 4

3. Two operations with parentheses

(16 / 4) + 44 = 48
(10 + 28) * 3 = 114
1 * (16 + 28) = 44

4. Three operations without parentheses

3 * 9 + 12 / 1 = 39
60 + 49 - 18 + 8 = 99
43 + 10 * 2 - 8 = 55

5. Three operations with parentheses

(132 / 12) + (46 - 15) = 42
(46 + 34) - (1 + 7) = 72
(21 + 27) * (14 - 7) = 336

6. Decimal arithmetic

0.5 * 0.5 = 0.25
1 / 10 = 0.1
7 / 2 = 3.5

Example Outputs with inference.py

Example 1 — Raw arithmetic prompt

python3 inference.py \
  --model WhirlwindAI/Arithmetic-SLM \
  --prompt "59 + 45 =" \
  --max-new-tokens 32 \
  --temperature 0.6 \
  --top-k 50 \
  --top-p 0.97 \
  --print-full

Expected style:

59 + 45 = 104

Example 2 — Production /no think format

python3 inference.py \
  --model WhirlwindAI/Arithmetic-SLM \
  --prompt "0.5 * 0.5 =" \
  --no-think \
  --max-new-tokens 48 \
  --temperature 0.6 \
  --top-k 50 \
  --top-p 0.97 \
  --repetition-penalty 1 \
  --frequency-penalty 0.0 \
  --no-repeat-ngram-size 0 \
  --seed -1 \
  --print-full

Example output:

[IM_START]user
0.5 * 0.5 = /no think[IM_END]
[IM_START]assistant
<think>
</think>
0.5 * 0.5 = 0.25[IM_END]

Example 3 — Operator priority

python3 inference.py \
  --model WhirlwindAI/Arithmetic-SLM \
  --prompt "8 * 5 + 4 / 4 =" \
  --no-think \
  --max-new-tokens 48 \
  --temperature 0.6 \
  --top-k 50 \
  --top-p 0.97 \
  --print-full

Expected style:

8 * 5 + 4 / 4 = 41

Example 4 — Parentheses

python3 inference.py \
  --model WhirlwindAI/Arithmetic-SLM \
  --prompt "(85 - 45) + 56 =" \
  --no-think \
  --max-new-tokens 48 \
  --temperature 0.5 \
  --top-k 40 \
  --top-p 0.95 \
  --print-full

Expected style:

(85 - 45) + 56 = 96

Example 5 — Three-operation expression

python3 inference.py \
  --model WhirlwindAI/Arithmetic-SLM \
  --prompt "3 * 9 + 12 / 1 =" \
  --no-think \
  --max-new-tokens 48 \
  --temperature 0.4 \
  --top-k 20 \
  --top-p 0.85 \
  --print-full

Expected style:

3 * 9 + 12 / 1 = 39

Next Research Directions

We will continue improving our dataset engineering, but more importantly, we want to teach the model what most models are never explicitly taught:

• —Binary calculation: Neural Application Binary Interface, or NABI, with 16-bit numerical structures, including floats.
• —FP16 to BASE-65,536 conversion: a float16 value is represented by 2 bytes, meaning 65,536 possible bit patterns. Base 65,536 also contains 65,536 possible integer values, making exact bit-level mapping possible.
• —Dot-product learning: explicit learning of scalar products on float16 vectors with 16, 8, 4, and 2 dimensions.
• —Learning the dynamics of its own learning: training the model to predict its own weights and gradients over time, including its own gradient descent dynamics.

This project does not claim to be a revolution.

It is an experiment in making small models learn precise arithmetic, numerical structure, and eventually parts of their own learning dynamics.

By Science AND FOR SCIENCE <3