AI Voice Agent Architecture: What I Learned Building the Same Agent Three Times
I built the same production voice agent three times. The orchestrator collapsed under coupling, server-gated turns created dead air, and the third architecture — where the realtime model owns the conversation — is the one that survived. Pros, cons, and diagrams of all three.