Steering Pretrained Drafters during Speculative Decoding

This research addresses the critical bottleneck of drafter-verifier misalignment in speculative decoding by introducing a dynamic alignment mechanism using s...

Level: advanced

By Frédéric Berdoz, Peer Rheinboldt, Roger Wattenhofer

Category: education