AdaFRUGAL: Adaptive Memory-Efficient Training with Dynamic Control

AdaFRUGAL introduces an adaptive optimization framework that dynamically adjusts subspace ratios and update frequencies to reduce GPU memory usage by 38% whi...

Level: advanced

By Quang-Hung Bui, Anh Son Ta

Category: research