VampNet

VampNet approaches music generation through masked acoustic token modeling: audio is encoded into discrete tokens across hierarchical codebooks, portions are masked, and a non-autoregressive transformer fills them back in. That mechanism makes it unusually flexible for a music model, since the same network handles unconditional generation, infilling missing sections of a track, and producing variations that riff on existing material, with per-codebook control for fine-grained edits. The release pairs a coarse model with a coarse-to-fine refinement stage and includes PyTorch training and fine-tuning scripts for custom datasets, a Gradio web interface, and Unloop, a Max/MSP integration for real-time experimentation. Created by researcher Hugo Flores Garcia, the code is on GitHub with pretrained checkpoints on Hugging Face; both carry Creative Commons non-commercial licenses, so usage is limited to research and personal projects. Music producers exploring generative sound design and audio researchers studying masked modeling are its main audience.