PyTorch-based end-to-end speech toolkit covering ASR, TTS, ST, and more; choose it if you want a non-Paddle implementation with broad community support.
End-to-End Speech Processing Toolkit
- 9.9k
- Python
- Apache-2.0
speech toolkit (ASR/TTS/SV/ST/KWS)
PaddleSpeech is an open-source toolkit on PaddlePaddle covering speech recognition, text-to-speech, speaker verification, speech translation, keyword spotting, and other audio tasks.
Same problem, same approach. Swapping one for another is a config change, not a rewrite.
PyTorch-based end-to-end speech toolkit covering ASR, TTS, ST, and more; choose it if you want a non-Paddle implementation with broad community support.
End-to-End Speech Processing Toolkit
PyTorch speech toolkit with modular recipes and many tasks; choose it for flexibility and active development.
A PyTorch-based Speech Toolkit
Speech AI toolkit (ASR/TTS) in the NeMo ecosystem; choose for integration with other NVIDIA NeMo features.
A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)
Solves the same problem with a different architecture or at a different layer. Expect to rewrite the integration.
C++ speech toolkit for voice agents with very low latency; choose for edge/real-time interfaces.
Very low latency speech to text, intent recognition, and text to speech, for building voice agents and interfaces
All-in-one simultaneous ASR/ST/TTS model; choose for simultaneous speech tasks in one model.
StreamSpeech is an “All in One” seamless model for offline and simultaneous speech recognition, speech translation and speech synthesis.
An earlier or more limited way to do the job. Still the right call when you need something small, proven, or CPU-only.
Production-focused streaming ASR toolkit, narrower than PaddleSpeech but great for low-latency ASR deployment.
Production First and Production Ready End-to-End Speech Recognition Toolkit
Expressive TTS model only, unmaintained; useful only for research on that model.
Foundational model for human-like, expressive TTS
Few-shot voice cloning TTS with strong Chinese support; choose when you need high-quality cloning from 1 minute of voice.
1 min voice data can also be used to train a good TTS model! (few shot voice cloning)
Voice cloning from 5-second samples; choose for real-time cloning demos, though it is narrower and older.
Clone a voice in 5 seconds to generate arbitrary speech in real-time
Mature TTS-only library, unmaintained for 2 years; use if you need a TTS-only solution and can maintain it yourself.
🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production
Voice cloning TTS, narrower scope; a classic project but not a full toolkit.
🚀Clone a voice in 5 seconds to generate arbitrary speech in real-time
High-quality multilingual TTS model; narrower than PaddleSpeech but a strong choice for TTS-only pipelines.
VoxCPM2: Tokenizer-Free TTS for Multilingual Speech Generation, Creative Voice Design, and True-to-Life Cloning
ASR-focused toolkit with streaming, VAD, punctuation and diarization; choose for production ASR pipelines.
Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
Audio/music/speech generation toolkit, no ASR; choose for synthesis research and production.
Amphion (/æmˈfaɪən/) is a toolkit for Audio, Music, and Speech Generation. Its purpose is to support reproducible research and help junior researchers and engineers get started in the field of audio, music, and speech generation research and development.
Voice cloning TTS for 600+ languages; choose for multilingual cloning.
High-Quality Voice Cloning TTS for 600+ Languages
Fast local neural TTS engine, archived; choose for on-device TTS without cloud.
A fast, local neural text to speech system
Multilingual ASR model with language ID and emotion; choose for compact ASR on Mandarin, Cantonese, English, etc.
Open-source SenseVoiceSmall model for Mandarin, Cantonese, English, Japanese, and Korean ASR, language ID, emotion recognition, and audio event detection.
Multi-voice TTS engine, unmaintained; use if you need its prompt-controlled TTS.
EmotiVoice 😊: a Multi-Voice and Prompt-Controlled TTS Engine
Reference implementation of VITS TTS; old, use for reproducing the paper.
VITS: Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech
YourTTS model code, old; use for zero-shot TTS research.
YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone
Old DiffGAN-TTS implementation; use only for research reproduction.
PyTorch Implementation of DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs
Old Speech-Transformer ASR implementation; use for educational purposes.
A PyTorch implementation of Speech Transformer, an End-to-End ASR with Transformer network on Mandarin Chinese.
These projects were analysed and named PaddleSpeech among their alternatives. The relationship is not symmetric — how PaddleSpeech rates them is a separate judgement, made when PaddleSpeech is analysed in its own right.
calls PaddleSpeech “A general speech toolkit with TTS capabilities; choose it when you work in PaddlePaddle or need ASR and speaker tasks alongside TTS in one framework.”
🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production
calls PaddleSpeech “Choose when you prefer the PaddlePaddle framework and want a comparable all-in-one speech toolkit with active maintenance.”
End-to-End Speech Processing Toolkit
calls PaddleSpeech “Choose when you need a full speech toolkit with streaming and Chinese-first ASR pipelines.”
Robust Speech Recognition via Large-Scale Weak Supervision
calls PaddleSpeech “Choose it for a comprehensive speech toolkit in PaddlePaddle if you are already in that ecosystem; it includes TTS and cloning in a different framework.”
Clone a voice in 5 seconds to generate arbitrary speech in real-time
calls PaddleSpeech “Choose when you prefer an actively maintained PaddlePaddle speech toolkit with voice cloning and broader ASR/TTS features over MockingBird's PyTorch pipeline.”
🚀Clone a voice in 5 seconds to generate arbitrary speech in real-time
calls PaddleSpeech “Choose when you need an all-in-one speech toolkit including TTS plus ASR, diarization, etc.”
SoTA open-source TTS
calls PaddleSpeech “choose when you are in the PaddlePaddle ecosystem and want a full ASR/TTS toolkit.”
Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
calls PaddleSpeech “choose when you are already in the PaddlePaddle ecosystem and want integrated ASR/TTS/speaker tooling.”
A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)
calls PaddleSpeech “Use when you want actively maintained SOTA ASR/TTS models and training pipelines in Python, and can trade on-device/ONNX portability for server-side depth.”
Speech-to-text, text-to-speech, speaker diarization, speech enhancement, source separation, and VAD using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Android, iOS, HarmonyOS, Raspberry Pi, RISC-V, RK NPU, Axera NPU, Ascend NPU, x86_64 servers, websocket server/client, support 12 programming languages
calls PaddleSpeech “Choose as a full speech toolkit (ASR, TTS, etc.) to build custom pipelines programmatically instead of using a ready webui.”
Gradio WebUI for creators and developers, featuring key TTS (Edge-TTS, kokoro) and zero-shot Voice Cloning (E2 & F5-TTS, CosyVoice), with Whisper audio processing, YouTube download, Demucs vocal isolation, and multilingual translation.
calls PaddleSpeech “PaddlePaddle-based speech toolkit with speech translation, if you prefer a broader toolkit over a single model.”
Foundational Models for State-of-the-Art Speech and Text Translation
calls PaddleSpeech “Choose when you want a full PaddlePaddle speech toolkit with TTS, ASR, and more; heavier framework.”
Use Microsoft Edge's online text-to-speech service from Python WITHOUT needing Microsoft Edge or Windows or an API key
calls PaddleSpeech “Choose when you work in PaddlePaddle and need an easy all-in-one speech pipeline.”
A PyTorch-based Speech Toolkit
calls PaddleSpeech “Choose it when you need a Python-native speech toolkit to embed ASR/TTS directly into your own application.”
VoiceStudio is the open-source, fully-local ElevenLabs alternative — voice cloning, voice design, video dubbing, dictation, transcription & audiobook creation in 646 languages.
calls PaddleSpeech “choose when you want an easy-to-use all-in-one speech toolkit with SOTA ASR/TTS and a Chinese-friendly stack.”
Very low latency speech to text, intent recognition, and text to speech, for building voice agents and interfaces
calls PaddleSpeech “when you prefer PaddlePaddle toolkit covering ASR/TTS/translation; different framework”
Real-time, local speech-to-text with streaming ASR, speaker diarization, translation, and OpenAI/Deepgram-compatible APIs.
calls PaddleSpeech “Comprehensive speech toolkit; choose it for extra features like TTS and speaker verification in one Python package.”
High-performance GPGPU inference of OpenAI's Whisper automatic speech recognition (ASR) model
calls PaddleSpeech “Choose when you need a full speech toolkit including ASR and translation alongside TTS, from the PaddlePaddle ecosystem.”
Amphion (/æmˈfaɪən/) is a toolkit for Audio, Music, and Speech Generation. Its purpose is to support reproducible research and help junior researchers and engineers get started in the field of audio, music, and speech generation research and development.
calls PaddleSpeech “Use for an all-in-one speech toolkit with TTS, ASR, and more, if you need broader speech tasks.”
High-Quality Voice Cloning TTS for 600+ Languages
calls PaddleSpeech “choose if you want an all-in-one speech toolkit with TTS/ASR on PaddlePaddle.”
A TTS that fits in your CPU (and pocket)