At a wedding or a party, there is usually a TV or a projector doing nothing. We wanted it to show every guest's photos as they are taken. That is the "live slideshow wall" in Fondays, a shared photo album app for events, and it turned out to be a nice small exercise in keeping real-time features honest.
Here are the rules we ended up with.
1. The socket is a doorbell, not the record
The wall opens a WebSocket and gets a short message whenever a photo lands. The message is a notification. The photos themselves always come from the normal REST API.
That one decision removes a whole class of bugs. If the socket drops for ten seconds, or a message is lost, the client refetches over REST and is correct again. Nothing has to be replayed, and a failed publish never fails the upload that caused it.
2. Fan out locally, widen with Redis
We run more than one API replica, and the phone uploading a photo is rarely connected to the same replica as the TV.…