Menu

Getting the V100 tensor cores back from Triton
πŸ“°

Getting the V100 tensor cores back from Triton

DEV Community: machinelearningΒ·Hocine BenkelayaΒ·6 days ago
#yeffkAIS
#devtomachinelearning#education#devto#dev#neurobrix
Reading 0:00
15s threshold

A lot of serious GPU capacity still sits in V100s: university clusters, second-hand servers, cloud instances priced down. They have tensor cores. Triton, on that architecture, does not use them.

This post is about how nbxtensor β€” the tensor library under NeuroBrix's Triton engine β€” turns them back on, and why it only does so where a measurement says it helps.

The gap

Triton is the kernel language NeuroBrix's Triton engine is written in. One kernel source serves NVIDIA, AMD, Intel and Apple GPUs, which is exactly why we use it. On Volta (sm_70), however, a matrix product written with tl.dot compiles to FMA instructions: ordinary fused multiply-adds on the CUDA cores.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.