nuemaan/skewadam

Tiered optimizer state allocation for memory efficient MoE training. Cuts optimizer memory by 97.4%, outperforming AdamW/Muon/Lion while fitting a 6.78B MoE on a single 40GB GPU.

2 best points · 0 comments · 1 Hacker News post · 14 GitHub stars · View on GitHub

Hacker News discussions

• Show HN: SkewAdam – A tiered optimizer that cuts MoE state memory by 97% (2 points, 0 comments, by anonnumaan, 2026-07-22)