RVC (Retrieval-based Voice Conversion)

Short for Retrieval-based Voice Conversion, RVC converts the timbre of one voice into another and can be trained from as little as ten minutes of clean speech. Its core technique is replacing input features with top-1 retrieval against an index of training-set features, which limits tone leakage from the source speaker; HuBERT handles feature extraction, a HiFi-GAN based vocoder synthesizes the audio, and the RMVPE algorithm extracts pitch reliably even for high vocal ranges. A Gradio web UI covers dataset preparation, training, and inference, and a separate real-time interface reaches around 170ms end-to-end latency, or roughly 90ms with ASIO hardware. Inference runs on NVIDIA GPUs, on AMD and Intel hardware, and via DirectML on Windows, with extras like model fusion for blending timbres and built-in vocal separation. The open source project is popular with musicians, streamers, and hobbyists producing voice covers and real-time voice changing.