Modelpublic

daviskas1/roora-v1-math

sourceHugging Facemitupdated 16d agoView on Hugging Face

0likes34downloads

Model Card

Roora-V1-Math

Roora-V1-Math is a very small decoder-only Transformer specialized in arithmetic generation. It is the mathematical component of the planned Roora-V1 family.

Model at a glance

Property	Value
Parameters	4,774,400 (~4.77M)
Hidden size	256
Transformer blocks	6
MLP size	1024
Context length	64 tokens
Vocabulary	37 characters
Precision	FP32
Training steps	30,000
Training data	Hundreds of thousands of automatically generated synthetic math examples
Checkpoint	`roora-math(1).pt`

What it can do

For its size, Roora-V1-Math shows surprisingly strong behavior on small and medium arithmetic patterns. Example generations observed from the released checkpoint include:

text

17^3=4913
33+37+29=99
2+3=5
12+34=46
37+48=85

The model is not a general-purpose calculator and does not guarantee exact answers for arbitrary large integers or long multi-step expressions. Its behavior becomes less reliable as number ranges and expression complexity increase.

Architecture

Roora-V1-Math uses a compact decoder-style Transformer with learned token and positional embeddings, pre-normalized self-attention blocks, MLP blocks, a final LayerNorm, and a linear language-model head.

The checkpoint contains a 37-symbol character vocabulary. This makes the model intentionally tiny and easy to experiment with, but also limits its representational range compared with subword-tokenized LLMs.

Training

The model was trained for 30,000 steps on Google Colab hardware using a large collection of automatically generated synthetic arithmetic examples. The synthetic-data approach makes it possible to generate a very large number of clean, automatically labeled training examples without manual annotation.

Training details such as optimizer, learning rate schedule, batch size, exact hardware, and random seed are not included in this release because they are not encoded in the supplied checkpoint.

Files

—roora-math(1).pt — original PyTorch checkpoint.
—Roora-V1-Math-F32.gguf — FP32 GGUF container containing the model weights and Roora-specific metadata.

Intended use

This model is primarily an educational and experimental research project: small-model training, synthetic-data generation, arithmetic reasoning experiments, and development of the Roora architecture.

Limitations

Roora-V1-Math is only about 4.77M parameters. It should not be compared directly with modern billion-parameter language models. It can make arithmetic mistakes, especially outside the distributions represented during training.

Do not use its output as a source of truth for safety-critical, financial, medical, or other high-stakes calculations.

Roadmap

The broader Roora project is planned to include:

—Roora-Math — arithmetic specialist (this model)
—Roora-Chat — dialogue specialist
—Roora-V1 — a future unified Roora release

Credits

Created as part of the Roora project.