Entropy and Attention Dynamics in Small Language Models: A Trace-Level Structural Analysis on the TruthfulQA Benchmark
This research dissects the internal mechanics of Small Language Models, revealing how entropy and attention dynamics dictate truthfulness and hallucination r...