optimizer
optimizer_resultsembedding-optimizer-study-analysis-artifacts
Embedding optimizer study analysis artifacts
This repository preserves analysis artifacts for the current DenseOn comparison
of AdamW, Muon and NorMuon. The source repository and paper
contain the completed experiments, protocols, exact analysis and restoration tools.
Model and optimizer states are in the separate
checkpoint repository.
Current scientific artifacts
Use the immutable revisions and manifests in these guides, not a broad download
of the mixed-history… See the full description on the dataset page: https://huggingface.co/datasets/qcz/embedding-optimizer-study-analysis-artifacts.lora-optimizer-community-cache
LoRA Optimizer — Community Cache
Shared analysis results for the LoRA Optimizer ComfyUI node.
LoRA merge analysis is hardware-agnostic — the same LoRA files always produce the same conflict metrics and optimal merge config regardless of GPU tier. This dataset lets users share and reuse those results so nobody has to run the AutoTuner from scratch.
How It Works
The AutoTuner computes pairwise conflict metrics (cosine similarity, sign conflicts, subspace overlap)… See the full description on the dataset page: https://huggingface.co/datasets/ethanfel/lora-optimizer-community-cache.model-optimizer-queuetool-optimizer-datasetautoresearch-optimizer-findings
Ablating Optimizer Ideas Around MuonAdamW for Transformer Pretraining
Overview
27 optimizer experiments on a 50M-parameter transformer, systematically testing whether anything can beat a well-tuned MuonAdamW baseline. We tried curvature-aware momentum, spectral boosting via power iteration, dual-timescale EMAs, variance-adaptive scaling, sign-coherence weighting, and more — drawing from recent papers (NAMO, AdEMAMix, Muon-VS, ROOT) and first-principles linear algebra.… See the full description on the dataset page: https://huggingface.co/datasets/mishig/autoresearch-optimizer-findings.