Adrian Tam
Author ProfileClaim This Author Profile
Prove ownership by publishing #HashtagPLUS and this profile link on your author page or an article under your byline. A moderator or admin will review the request before it merges into your real HashtagPLUS username.
π machinelearningmastery.comSource
Overview This post is divided into five parts; they are: β’ Asymmetric Affine Quantization β’ Post-Training Quantization with PyTorch β’ GPTQ and AWQ β’ Quantizing a Model to GGUF β’ Evaluating Weight-Only Quantization Note: The code examples in this post use TorchAO and require a CUD
π machinelearningmastery.comSource
A transformer layer is built from familiar operations: linear projections, attention, normalization, activation functions, and residual additions. A direct implementation may launch a separate accelerator kernel for every operation and write every intermediate tensor to device me
π machinelearningmastery.comSource
Overview This post is divided into eight parts; they are: β’ The Checkpoint Fixes the Attention Architecture β’ A Closer Look at Attention Memory β’ Multi-Head Attention β’ Multi-Query and Grouped-Query Attention β’ Sliding-Window and Mixed Attention β’ Multi-Head Latent Attention β’ Wh
π machinelearningmastery.comSource
Overview This post is divided into five parts; they are: β’ How Floating-Point Numbers Work β’ Common Precision Formats β’ Loading a Model at a Different Precision β’ Mixed Precision and Sensitive Operations β’ Measuring the Result How Floating-Point Numbers Work A floating-point numb