j0no12/nero-optimizer-work-m-simow-b0p5-lr0p006

Nero Optimizer Work — M-SimOW (beta=0.5, lr=0.006)

This repository contains the final MLX checkpoint for the M-SimOW (beta=0.5, lr=0.006) arm of the Nero Optimizer Work research sweep.

Summary

• —Optimizer: m_simow
• —Requested learning rate: 0.006
• —Momentum beta: 0.5
• —Training budget: 500,000,000 tokens
• —Context length: 128 tokens
• —Backend: Apple MLX
• —Model family: matched dense-deep decoder, 2,048-token vocabulary, 128-wide residual stream, 6 blocks, 32-dimensional attention heads, 148-wide gated MLP
• —Stored model parameters: approximately 999,680
• —Final training loss: 3.365410
• —Final logged throughput: 351,982 tokens/s
• —Tail throughput: 351,982 tokens/s (median of the final logged samples)
• —Final tokens seen: 500,000,000

Files

• —model.npz — final MLX model weights from checkpoint_000500000000
• —state.json — checkpoint state metadata
• —run.json — frozen run configuration
• —metrics.jsonl — complete training metrics log
• —config.json — model and publication metadata

Evaluation

The figures above are training-run measurements. An independent held-out validation artifact was not saved with these runs, so this card does not claim a validation score. Compare checkpoints using the same frozen evaluation pass before drawing quality conclusions.

Intended use and limitations

This is an experimental research checkpoint, not an instruction-tuned or production-ready language model. It is published to make the optimizer comparison reproducible. The raw MLX weights require a compatible local MLX loader and are not a Transformers checkpoint.

Reproducibility

All arms use the same prepared finephrase-balanced-500m-2k-v2 token stream, 128-token context, 32-example batches, and 500M-token target. The full local training log is included in metrics.jsonl.

License

No new model license is asserted by this experimental publication. Review the source-data terms before redistribution or downstream use.