1. Compute and Power Constraints

Mobile CPUs/GPUs are orders of magnitude slower than datacenter hardware. Running complex models continuously can overheat devices and deplete batteries. For example, on-device object detection may take tens of milliseconds per frame, far slower than server GPUs. One case study found that enabling GPU delegation reduced inference from ~28 ms to ~11 ms per image, illustrating how hardware accelerators help but only partially bridge the gap. In practice, heavy inference slows the UI and degrades UX. Developers must mitigate this by using quantized (INT8) or pruned models, leveraging mobile NN accelerators (NNAPI on Android, Metal on iOS), and limiting workload (e.g. reducing camera frame rate or downscaling images). Offloading to edge/ cloud (e.g. sending data to a server) avoids local load but adds network latency. In sum, AI features that demand high throughput or low latency (e.g. real-time video analysis) are often beyond pure on-device execution.

2.…