I run a photo enhancer that does super-resolution fully in the browser: TensorFlow.js on WebGL, nothing is uploaded. The first version used the upscaler library (UpscalerJS). On a 1.2 megapixel photo it took 60-110 seconds and froze the page for about 40 seconds. A user told me "it does not accept any of my photos", which was partly a bad 0.6 MP limit on my side and partly the freeze. Here is what I found while fixing it. Most of it applies to any browser-side ML on WebGL.

1. Every new tensor shape recompiles the shaders

tfjs compiles WebGL programs per operation and per tensor shape . On my machine (an M2 Max) one new shape cost 8-17 seconds of shader compilation. The killer was the edge tiles: I cut the image into tiles, and the last row and column had a different size, so the model was compiled again for them.

What fixed it:

Make every tile exactly the same shape.…