Adding voice to a mobile app sounds like a solved problem until you ship one. A recognizer that works beautifully in a quiet-room demo turns flaky on a crowded bus, partial transcripts flicker in and out of existence, and users end up repeating themselves until they give up. I learned this building Fonetic AI , an app that helps people learn English by listening to their pronunciation, running real-time conversational practice, and giving actionable feedback on their mistakes. The frontend is Flutter and Dart, the speech work goes through AI speech APIs with Firebase on the backend, and the app ships more than 100 interactive lessons where every spoken answer is scored in real time.

This article distills the architecture that made real-time speech feedback actually work in production: a partial-results pipeline built on the speech_to_text package, word-level scoring against an expected phrase, an explicit session state machine, and the permission and performance pitfalls that will waste your week if you…