VoiceCraft

Created by Puyuan Peng, David Harwath and co-authors, VoiceCraft treats speech as a sequence of neural codec tokens and fills in missing spans, which lets one model both rewrite words inside an existing recording and synthesize new speech in a voice it has never heard, given a reference clip of several seconds. Released checkpoints come in 330M and 830M sizes, including TTS-tuned versions trained on GigaSpeech and part of LibriLight, which expect prompt plus output to stay under about 16 seconds. Try it through a local Gradio app, Jupyter notebooks, command-line demo scripts, Docker with the NVIDIA container toolkit, Colab or a Hugging Face Space, and training code plus the RealEdit dataset are included. The environment pins older components (Python 3.9, PyTorch 2.0.1, CUDA 11.7, Montreal Forced Aligner and espeak-ng), and the last code change landed in March 2025, so expect some setup friction. Licensing rules out commercial work: code uses a Creative Commons non-commercial share-alike license and weights use the Coqui Public Model License. Suited to researchers studying zero-shot cloning and edits to recorded speech.