YanaiEliyahu/AdasOptimizer

ADAS is short for Adaptive Step Size, it's an optimizer that unlike other optimizers that just normalize the derivative, it fine-tunes the step size, truly making step size scheduling obsolete, achieving state-of-the-art training performance

1 best points · 1 comments · 1 Hacker News post · 86 GitHub stars · View on GitHub

Hacker News discussions

• Optimizer obsoletes step-size scheduling, 100% on MNIST's training set 11 epochs (1 points, 1 comments, by YanaiEliyahu, 2021-01-07)