This research identifies a critical failure mode in decoder-only transformers called 'runway cascade,' where indirect paths distort attention. It introduces ...
Level: advanced
By Hunjae Lee, Corey Clark
Category: research