AxionLab-Co/MiniAxion1-0.9M

🧠 MiniAxion1-0.9M

MiniAxion1-0.9M is a Nano Reasoning Model (NRM) with ~920K parameters designed to explore the emergence of structured reasoning in extremely small neural networks.

Despite its minimal size, the model demonstrates strong consistency in reasoning format and step-based thinking using explicit <THINK> and <STEP> tokens.

🚀 Overview

• —Model Type: Nano Reasoning Model (NRM)
• —Parameters: ~920,833
• —Architecture: Transformer (6 layers: 2 entry + 2 shared + 2 exit)
• —d_model: 256
• —Heads: 8
• —FFN size: 512
• —LoRA Rank: 16
• —Vocabulary Size: 2048
• —Training Time: ~80 minutes (CPU)

🧠 Key Capabilities

✅ Structured Reasoning

The model reliably produces structured reasoning traces:

<THINK>
<STEP> ...
<STEP> ...
</THINK>
<ANS>...</ANS>

• —100% usage of reasoning tokens
• —Consistent multi-step formatting
• —Stable output structure across tasks

⚡ Ultra-Lightweight

• —Runs efficiently on CPU
• —Designed for experimentation and rapid iteration
• —Suitable for embedded or game-like environments

🧪 Research-Oriented Design

MiniAxion1 is not intended to compete with large-scale models. Instead, it is built to:

• —Study reasoning emergence in small models
• —Explore structure vs correctness trade-offs
• —Enable fast iteration cycles for AI research

📊 Evaluation Results

Average Accuracy: 16.9%

🔍 Observations

• —The model learns reasoning structure before reasoning correctness
• —Chain-of-thought formatting is highly reliable
• —Arithmetic and symbolic reasoning remain limited at this scale
• —Evidence of partial decoupling between reasoning steps and final answers

⚠️ Limitations

• —Weak performance on arithmetic and multi-step reasoning tasks
• —Susceptible to incorrect intermediate reasoning steps
• —Limited generalization beyond trained patterns
• —Not suitable for production use in critical systems
• —Due to 920k parameters, low results on evaluation is expected

🎯 Intended Use Cases

• —🧪 AI research and experimentation
• —🎮 Game AI / NPC reasoning simulation
• —📚 Educational demonstrations of reasoning structure
• —⚙️ Lightweight reasoning prototypes

Quick start

import torch
from model import NRMModel
from tokenizer import Tokenizer

# load
model = NRMModel.from_config("config.json")
model.load_state_dict(torch.load("model.pt"))
model.eval()

tokenizer = Tokenizer.load("tokenizer.json")

def generate(prompt):
    tokens = tokenizer.encode(prompt)
    output = model.generate(tokens)
    return tokenizer.decode(output)

print(generate("<INST>What is 2 + 2?</INST>"))

🧠 Philosophy

MiniAxion1 explores a key question:

Can structured reasoning emerge in extremely small models?

This model provides early evidence that:

• —Reasoning format can be learned efficiently
• —Structure and correctness are separable capabilities
• —Useful behavior can emerge even at sub-1M scale

🔮 Future Directions

• —Improved dataset alignment for arithmetic reasoning
• —Scaling parameters (1M → 10M range)
• —Better coupling between reasoning and answers
• —Task-specific specialization (e.g., math-only variants)
• —distillation knowledge on bigger models

🤝 Acknowledgments

This model was developed as part of ongoing experimentation in nano-scale reasoning systems. the main question was: "How low could a model think(or mimic it)?

📎 Model

👉 https://huggingface.co/AxionLab-Co/MiniAxion1-0.9M

🧪 Disclaimer

This is an experimental research model. Outputs may be incorrect even when reasoning appears structured or convincing.