Modelpublic

aquilesfd/neo-3-1B-A90M-Base

sourceHugging Facemitupdated 8mo agoView on Hugging Face

8likes451downloads

Model Card

neo-3

![Paper](https://huggingface.co/aquiffoo/neo-3-1B-A90M-Base/blob/main/neo3Technical_Report.pdf)

This is the 1B-A90M-Base model. Check out the 3B-A400M-Base model and the post-trained version of your model.

The neo-3-1B-A90M and 3B-A400M models are the successor to neo-2-345M-C1 and C2, featuring:

—Pre-trained and post-trained models
—Context windows of 8K for 1B and 32K for 3B
—The first ever neo reasoning model
—Mixtral architecture

The post-trained models will release January 3rd, 2026. The pre-trained models have released on December 31st, 2025. A technical report is on the works, but training data is fully available for replication.

This model has an MIT license.

Evaluations for base models

Performance

	MMLU	HellaSwag	PIQA	ARC avg	GSM8K	Avg.
neo-3-1B-A90M	32.7	52.3	63.4	42.1	2.2	38.54
neo-3-3B-A400M	42.1	59.9	67.5	50.6	5.7	45.16
SmolLM2-360M	32.7	52.3	63.4	42.1	2.2	38.54
Gemma 3 270M	26.5	40.9	67.7	43.4	1.1	35.92
Qwen3-0.6B-Base	44.0	55.3	60.9	52.4	49.7	52.46

Efficiency

	Active Parameters	Training Tokens	Avg. p/ B active	Avg. p/ T training tokens
neo-3-1B-A90M	120M	1.2T	321.17	32.12
neo-3-3B-A400M	380M	1.2T	118.84	37.63
SmolLM2-360M	362M	4T	106.46	9.64
Gemma 3 270M	268M	6T	134.03	5.99
Qwen3-0.6B-Base	642M	36T	81.72	1.46

Charts

Task list

—[x] Train neo-3-Base models
—[x] Train neo-3 Instruct/Thinking models
—[x] Release neo-3-Base models
—[ ] Release neo-3 Instruct/Thinking models
—[ ] Publish neo-3 Technical Report
—[ ] Train neo-3-VL Base model
—[ ] Train neo-3-VL Instruct model
—[ ] Release neo-3-VL Base model
—[ ] Release neo-3-VL Instruct model
—[ ] Publish neo-3-VL Technical Report
—[ ] Train neo-3.1-Base models
—[ ] Train neo-3.1 Instruct/Thinking/VL-Instruct models
—[ ] Release neo-3.1-Base models
—[ ] Release neo-3.1 Instruct/Thinking/VL-Instruct models
—[ ] Publish neo-3.1 Technical Report