I spent the last few months building SkyMeetAI — a video conferencing platform that watches participants' faces and listens to their voices in real time to figure out how they're feeling. Not via surveys. Live, during the call, every 150–500ms.

Redis returned null into a socket event handler at 4 AM once, and my participant list had quietly vanished. No error. No trace. Just... gone.

That happened more than once. This post is about what that actually took to build — the distributed systems problems, the browser media weirdness, the ML decisions, and the bugs I'm documenting anyway because that's how this stuff works.

Live demo : skymeetai.onrender.com (live preview may not be available due to resource constraints) | Repo : github.com/AnupamKumar-1/skymeetAI

Stack

Frontend : React + WebRTC ( RTCPeerConnection , AudioWorklet , MediaRecorder )

Backend : Node.js + Express + Socket.IO + @socket.io/redis-adapter + MongoDB

Emotion service : Python FastAPI + Socket.IO + MediaPipe + Wav2Vec2 + PyTorch +…