j0no12/nero-optimizer-work-simo

Nero Optimizer Work — SimO

This repository contains the final MLX checkpoint for the SimO arm of the Nero Optimizer Work research sweep.

Summary

• —Optimizer: simo
• —Requested learning rate: the run's configured default/controller schedule
• —Training budget: 500,000,000 tokens
• —Context length: 128 tokens
• —Backend: Apple MLX
• —Model family: matched dense-deep decoder, 2,048-token vocabulary, 128-wide residual stream, 6 blocks, 32-dimensional attention heads, 148-wide gated MLP
• —Stored model parameters: approximately 999,680
• —Final training loss: 4.859327
• —Final logged throughput: 420,824 tokens/s
• —Tail throughput: 420,811 tokens/s (median of the final logged samples)
• —Final tokens seen: 500,000,000

Files

• —model.npz — final MLX model weights from checkpoint_000500000000
• —state.json — checkpoint state metadata
• —run.json — frozen run configuration
• —metrics.jsonl — complete training metrics log
• —config.json — model and publication metadata

Evaluation

The figures above are training-run measurements. An independent held-out validation artifact was not saved with these runs, so this card does not claim a validation score. Compare checkpoints using the same frozen evaluation pass before drawing quality conclusions.

Intended use and limitations

This is an experimental research checkpoint, not an instruction-tuned or production-ready language model. It is published to make the optimizer comparison reproducible. The raw MLX weights require a compatible local MLX loader and are not a Transformers checkpoint.

Reproducibility

All arms use the same prepared finephrase-balanced-500m-2k-v2 token stream, 128-token context, 32-example batches, and 500M-token target. The full local training log is included in metrics.jsonl.

License

No new model license is asserted by this experimental publication. Review the source-data terms before redistribution or downstream use.