A virtual try-on needs two pictures: a photo of the person and a photo of the garment. The image endpoint we run our standard model on accepts exactly one input image. We didn't want to change that endpoint, or the API contract our mobile app depends on, just to add a feature to the website.
The workaround turned out to be simple, and one detail in it made the difference between a feature that works most of the time and one that works every time we tried it.
Step 1: put both photos in one image, in the browser
The client builds a single image: the person on the left, the garment on the right on a plain light background, like a product shot. The pair is 1024 pixels tall, so the person keeps the same resolution as a normal single-photo edit.
The tricky part is the shape. The edit model only produces a fixed set of aspect ratios (1:1, 4:5, 3:4, 2:3, 9:16 and their landscape versions). If the collage doesn't match one of them exactly, the model pads or crops it, and the person ends up shifted or squeezed.…