I ship Phoenix Local AI , a private AI chat app for Android. You can connect it to a cloud model, but the point is that it can run a model on the phone itself, in airplane mode, with nothing leaving the device.
The on-device engine is LiteRT-LM , Google's runtime for .litertlm model files, called from Flutter. Getting it to answer at all was one job. Getting it to answer at a speed a person will wait for, on a phone that isn't a flagship, was another. These are my notes, measured on a POCO M2 Pro from 2020 over wireless adb, because a phone like that is a better test of "runs on a phone" than whatever's new this year.
1. Pin the runtime, and pin the models
The Flutter plugin I started from bundled a LiteRT-LM build, and the version matters more than you'd expect:
0.18 worked with my bindings.
0.19 is labelled "C API 1.0" and breaks the ABI. Nothing warns you; it just stops working.
So the app pins 0.18.…