Originally published on khadim.tech .

In short: On an RTX 5090 with TensorRT 10.12, a calibrated INT8 build of RF-DETR put zero layers in INT8. TensorRT's own timing chose FP16 for every layer, so the INT8 engine was the FP16 engine: about 5x faster than the PyTorch FP32 forward, with no extra gain. Forcing INT8 through Q/DQ nodes made it 17% slower and wrecked accuracy. On a CPU without VNNI, only PyTorch dynamic INT8 sped the forward up (1.3-2x), and it changed some detection decisions. Distilling RF-DETR Medium into Nano with pseudo-labels lowered accuracy on every seed. For RF-DETR on an RTX 5090, TensorRT FP16 was the answer, and NVIDIA's own release notes point the same way for DETR models.

I maintain a training and serving engine for RF-DETR defect-detection models.…