Hierarchical Balance Packing: Towards Efficient Supervised Fine-tuning for Long-Context LLM
Explore Hierarchical Balance Packing (HBP), a novel approach to optimize data packing and reduce communication overhead in long-context LLM training, achievi...