HiSpec introduces a hierarchical speculative decoding framework leveraging early-exit models to achieve significant throughput gains while maintaining accura...
Level: advanced
By Unknown
Category: research