#Mxfp8

As language models grow, scaling dense architectures becomes increasingly expensive. In a dense transformer, every token passes through every layer…