AlbertoFdezHdez/OUI

OUI Need to Talk About Weight Decay

📄 Read the paper on arXiv 🧠 Try the code on GitHub

🧪 Overview

We introduce OUI (Overfitting–Underfitting Indicator), a simple yet powerful diagnostic tool to monitor the training dynamics of deep neural networks and identify the optimal value for weight decay — without using a validation set.

Unlike traditional validation-based hyperparameter tuning, OUI:

• —Converges faster than metrics like accuracy or loss.
• —Signals overfitting and underfitting through the internal statistics of the model.
• —Provides a reliable early criterion for choosing the regularization strength during training.

📊 Results

OUI was validated across multiple vision benchmarks:

• —DenseNet-BC-100 on CIFAR-100
• —EfficientNet-B0 on TinyImageNet
• —ResNet-34 on ImageNet-1K

Across all experiments, keeping OUI in a specific target range consistently led to improved validation accuracy.

📝 Citation

If you find this work useful, please cite:

@article{fernandez2025oui, title={OUI Need to Talk About Weight Decay}, author={Fern{\'a}ndez-Hern{\'a}ndez, Alberto and Mestre, Jos{\'e} I. and Dolz, Manuel F. and Duato, Jose and Quintana-Ort{\'i}, Enrique S.}, journal={arXiv preprint arXiv:2504.17160}, year={2025} }

🤝 Acknowledgments

Developed by

Alberto Fernández-Hernández,

José I. Mestre,

Manuel F. Dolz,

Jose Duato,

Enrique S. Quintana-Ortí

Affiliations: Universitat Politècnica de València, Universitat Jaume I, Qsimov Quantum Computing. 📬 Contact

Feel free to reach out via GitHub Issues or open a discussion if you have questions, feedback, or suggestions! 💡 License

This project is released under the MIT License.

Let me know if you want a version with Hugging Face integration badges or a section for Colab demos.