bird-of-paradise/muon-tutorial

Understanding the Muon Optimizer: Theory and Implementation 📘 Contents Try It Yourself -- base and advanced implementions Introduction to Muon The Problem: Skewed Singular Values Newton-Schulz Orthogonalization Muon vs. AdamW When to Use Muon Implementation Details Practical Performance Analysis Optimization Techniques Lessons Learned Concludsion Extended Work 🧪 Try It Yourself The included Colab notebook allows you to run all experiments and… See the full description on the dataset page: https://huggingface.co/datasets/bird-of-paradise/muon-tutorial.