Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs

This research introduces a conditional scaling law that integrates architectural parameters like MLP-to-attention ratios into the Chinchilla framework, offer...

Level: advanced

By Unknown

Category: research