Motif-Technologies/optimizer

1# Muon Optimizer Test2 3This directory contains a test script for the **Muon optimizer**.4 5## Prerequisites6 7- **GPU Requirement**8  - All tests require **8 GPUs** by default.9  - If you have fewer GPUs available:10    - Modify the parallelism configurations in `test_muon.py`.11 12- **Model Access**13  - The tests require access to the private model repository:14    - `Motif-Technologies/Motif-2.6B-4layer-random` on Hugging Face.15  - Set your Hugging Face token via the environment variable `HF_TOKEN`.16  - If you don’t have access, please contact the maintainer.17 18- **Using a Different Model (Optional)**19  - You may modify the test to use a different model by:20    - Updating the model name in `conftest.py::inputs`.21    - Adjusting the tensor parallel rules in `utils.py::_apply_tp`.22 23## Usage24 25- To execute the test with 8 GPUs, simply run:26 27```bash28./run_test.sh29```30 31- To check the other available options, you can use:32 33```bash34pytest --help35...36Custom options:37  --measure-perf        Measure execution time and peak memory usage during optimizer step.38  --do-profile          Enable profiling during tests.39  --skip-verify         Skip verification of optimizer step correctness with sequential implementation.40                        This can be useful when GPU memory is limited.41...42```43