🖼️00Read later0Read MoreFlashAttention: Why Tiling Attention Makes Transformers Fast Without Changing the ResultDEV Community: deeplearning·Umesh Tharuka Malaviarachchi·4 days ago#RfA0icjg#devtodeeplearning#education#devto#dev#block+6 more🧰Tag tools✨Add tagTransformers did not get slow because attention needs too many multiplications. They got slow because...15s
🖼️00Read later0Read MoreThe 5.5 Cliff: The Bug in Every Softmax That Silently Zeroes Your Model's Best GradientDEV Community: machinelearning·Daniel Sam Pete Thiyagu·22 days ago#JqQJCKbf#devtomachinelearning#education#devto#dev#gradient+6 more🧰Tag tools✨Add tagEvery model you have ever trained runs through the same two functions: softmax turns raw scores into...15s
🖼️00Read later0Read MoreThe Softmax Bottleneck: Why Making LLMs Bigger Doesn't Always Make Them SmarterDEV Community·Vikrant Shukla·6 months ago#yvcaykPr#llm#ai#deeplearning#model#output+5 more🧰Tag tools✨Add tagWhen researchers scale a language model — more parameters, more layers, wider hidden dimensions —...15s
🖼️00Read later0Read MoreHow Transformer Attention Is ComputedDEV Community·Tawan Shamsanor·7 months ago#RbUfoUtN#ai#deeplearning#transformers#attention#token+5 more🧰Tag tools✨Add tagAttention doesn't actually look at all words. Here's how Transformer attention is computed step by step.15s
📰00Read later0Read MoreMaking Softmax More Efficient with NVIDIA Blackwell UltraNVIDIA Technical Blog·Jamie Li·7 months ago#qZ0MlVb0#x2d#agenticaigenerativeai#datacentercloud#cloudservices#blackwell+7 more🧰Tag tools✨Add tagLLM context lengths are exploding, and architectures are moving toward complex attention schemes like Multi-Head Latent Attention (MLA) and Grouped Query…15s