Serving tech enthusiasts for over 25 years.

TechSpot means tech analysis and advice you can trust .

Sounding off: A used RTX 3090 has become the go-to card for running AI models at home, mostly because 24GB of memory costs far less on a gaming card than on a workstation one. That bargain comes with a catch Nvidia prints in its own spec sheets, though. On GeForce cards, one of the most common kinds of AI math runs at exactly half speed. A few open-source projects now claim to remove that limitation on RTX 20-, 30-, and 40-series cards.

The limit lives in the tensor cores. When they multiply FP16 or BF16 numbers and add up the results in higher-precision FP32, GeForce cards issue those instructions at half the rate of the FP16-accumulate mode. That FP32 mode is the one most training and plenty of inference actually rely on.…