Motif-Technologies/optimizer
Merge pull request #34 from MotifTechnologies/fix/qk-clip-rotary-full-gamma
qk_clip: value-only fix — keep original structure, drop per-row rewrite [skip-build]
refactor(qk_clip): fold row-scale logic into one expand_row_scales helper [skip-build]
refactor(qk_clip): apply γ exponent per-row so q_pe→full γ in both paths [skip-build]
fix(qk_clip): scale q_pe rows by full γ, not γ^(1-k_ratio) [skip-build]
Merge pull request #32 from MotifTechnologies/feat/qk-clip-k-ratio
feat: add k_ratio for Q/K split control in MLA QK clipping [skip-build]
feat: CPUOffloadPool.reload_group에 sync_streams 파라미터 추가 (#31)
feat: extend QK-Clip to support MLA (MuonClip Algorithm 1) [skip-build] (#28)
Revert "fix: disable CUDA graphs in Newton-Schulz for cpu_offload compatibility" (#29)
Merge pull request #27 from MotifTechnologies/fix/cudagraph-offload-conflict
Add built binary [skip-build]
fix: disable CUDA graphs in Newton-Schulz for cpu_offload compatibility
Replace cpu_offload constructor param with turn_on/turn_off API (#26)
Merge pull request #25 from MotifTechnologies/fix/invalidate_cache_adamw
Invalidate AdamW tensor caches on load_state_dict [skip-build]
draft commit for cpu_offload (#23)
Replace toy PP tests with real-model-based pipeline tests [skip-build]
Add correctness verification to PP tests using fully_shard [skip-build]
Remove correctness check from PP tests, focus on deadlock detection [skip-build]
Add PP + dp_replicate deadlock regression tests [skip-build]
Update fast path comment to reflect current behavior [skip-build]
Update comment to reflect use_local_synchronization behavior [skip-build]
Fix deadlock in construct_shard_mesh with PP + dp_replicate > 1
Apply pre-commit formatting (isort) [skip-build]
Add MoE uneven shard test with mixed expert and non-expert params [skip-build]
Add uneven shard correctness test [skip-build]
Add optimization docs and update implementation guide [skip-build]
Update tests for MoE and parallel optimizations [skip-build]
Muon optimizer: expert batching, parallel caching, A2A overlap [skip-build]
Optimize pipeline: batched update, zero-copy scatter, prelaunch gather [skip-build]
Cache AdamW placement grouping and tensor lists [skip-build]
Add torch.compile, CUDA graph, and compiled momentum [skip-build]
Apply suggestions from code review
Add mhc_attn, mhc_ffn, lambda_proj to skip_keys
Remove verbose param_groups summary logging
Support multi-component expert_keys (e.g. "experts.w1")
Extract is_expert_param() helper to consolidate expert key matching
Include original (pre-normalize) FQN in is_muon logging
Add info-level logging for param group classification (Muon vs AdamW)
Use component-level matching for expert_keys to avoid shared_experts collision
Normalize parameter FQNs to handle torch.compile / checkpoint wrappers
Merge pull request #17 from MotifTechnologies/optimal-ns-coefficients
Apply pre-commit formatting (yapf) [skip-build]
Add max_iter cap and non-finite checks to _optimal_quintic [skip-build]
Apply pre-commit formatting (yapf, isort) [skip-build]
Add comment explaining _coeffs_list and Polar Express vs former NS [skip-build]
Replace hardcoded NS coefficients with analytically optimal ones [skip-build]
Refactor pipeline to async generator pattern (#16)
Support mHC (#15)