Everyone talks about deploying ML on edge devices. Very few people show what happens when you actually try.

I ran a full benchmark of four lightweight transformer models - DistilBERT, MobileBERT, TinyBERT-6L, and TinyBERT-4L — against traditional ML baselines on three real-world fault detection datasets.

The Setup

NASA C-MAPSS : Turbofan engine degradation (20,631 samples, 15% failure rate)

SECOM : Semiconductor manufacturing (1,567 samples, 6.6% failure rate)

UCI Predictive Maintenance : Industrial machine failure (10,000 samples, 3.4% failure rate)

All experiments ran on a T4 GPU with consistent hyperparameters.

The Results

Model F1 Size CPU Latency

XGBoost 87.9% 0.5 MB 0.002 ms

TinyBERT-4L 87.8% 55 MB 18 ms

DistilBERT 87.6% 255 MB 138 ms

MobileBERT: The Surprise Failure

MobileBERT — specifically designed for mobile deployment — scored 0% F1 on every dataset .…