Explore COSMOS, a novel hybrid adaptive optimizer designed to balance memory efficiency and performance in Large Language Model training through eigensubspac...
Level: advanced
By Unknown
Category: research