real-time voice/multimodal AI agent frameworks
Pipecat is an open-source Python framework for building real-time voice and multimodal AI agents by wiring together audio transports, STT, LLM, TTS, and orchestration.
An earlier or more limited way to do the job. Still the right call when you need something small, proven, or CPU-only.
when you need cross-platform voice/chat app support for Alexa/Google/Messenger and can accept archived, older code.
🔈 The React for Voice and Chat: Build Apps for Alexa, Messenger, Instagram, the Web, and more
when you need a traditional intent/entity/dialogue platform for deep-domain voice/chat rather than a live streaming media pipeline.
An Open Source Conversational AI Platform for Deep-Domain Voice Interfaces and Chatbots.
when you want a lightweight Ruby state-machine for text/telephony bots rather than a Python realtime multimodal agent stack.
An open source Ruby framework for text and voice chatbots. 🤖
when you want a prebuilt self-hosted voice-first agent platform and can tolerate 1.8 years of stalled development.
End-to-end platform for building voice first multimodal agents
Does the same job, but ships as an app. Useful to a person, not swappable into a codebase.
when you want a prebuilt self-hosted realtime voice assistant server rather than writing your own pipeline.
Real-time voice assistant — WebRTC streaming, faster-whisper ASR, local LLM, Vui Nano (300M) TTS. OpenAI Realtime API compatible. Voice cloning, barge-in, ~9× realtime on a 4090. Apache 2.0.
when you want a ready-to-run on-device GPT-Live-style assistant rather than a composable agent framework.
On-device, real-time multimodal AI with features similar to GPT-Live
when you prefer a visual no-code workflow builder with telephony/BYOK over coding your own agent framework.
Open source voice AI platform. Self-hosted alternative to Vapi and Retell. On Prem, BYOK across Speech to Speech or LLM/STT/TTS, with a visual workflow builder, MCP native and telephony support.
These projects were analysed and named pipecat among their alternatives. The relationship is not symmetric — how pipecat rates them is a separate judgement, made when pipecat is analysed in its own right.
calls pipecat “choose for realtime voice/multimodal agent pipelines using LLMs instead of intent-based NLU.”
💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants
calls pipecat “Choose it when you need realtime voice/multimodal pipelines instead of dashboard-authored text intents and channels.”
A python chatbot framework with Natural Language Understanding and Artificial Intelligence.