This research introduces Modern Hopfield Attention, a novel mechanism leveraging hidden states to stabilize deep Transformers by mitigating rank collapse and...
Level: advanced
By Tsubasa Masumura, Masato Taki
Category: research