I'm a solo developer, and for the past months I've been building Mangit : an iOS and Android app that translates manga, manhwa and comic pages. You take a photo or screenshot of a page, and the app puts the translation back inside the speech bubbles , so you read the page like it was lettered in your language.

Here's how it's built and what I learned.

The pipeline

Every page goes through five steps:

Detect the speech bubbles and text regions

Read the text (OCR)

Translate it, with the source language detected automatically

Erase the original lettering (inpainting) without damaging the art

Typeset the translation back into the same bubble, fitting the font size to the space

Steps 4 and 5 are where most of the work went.…