Menu

1 / 2

Raw WebSocket Voice Agent with AssemblyAI's Voice Agent API

DEV Community·Mart Schweiger·6 months ago
#gy9zBQcn
#how#voiceai#ai#session#tool
Reading 0:00
15s threshold

What "raw" means here

5-minute quickstart Raw WebSocket (this tutorial)

Lines of Python ~80

Events handled 6

Partial transcripts ( transcript.user.delta ) ✖

Tool calling ✖

Session resume on reconnect ✖

Speech start/stop logging ✖

Error code handling Minimal

If you want the fastest path to a working agent, start with the 5-minute quickstart. If you want to ship the Voice Agent API into a real product, build on this one — every edge case the protocol expresses is already in here.

Architecture

Microphone (sounddevice, 24 kHz PCM16) │ │ ┌──── client → server ────┐ │ │ session.update │ config (1st message) │ │ session.resume │ reconnect within 30s │ │ input.audio │ base64 PCM16 chunks │ │ tool.result │ send on next reply.done │ └────────────────────────┘ ▼ wss://agents.assemblyai.com/v1/ws ▲ │ ┌──── server → client ────┐ │ │ session.ready │ save session_id │ │ session.updated │ │ │ input.speech.started │ │ │ input.speech.stopped │ │ │ transcript.user.delta │ partial — live transcript │ │…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.