YuE

HKUST, the M-A-P community and partner labs have rebuilt this repository around YuE2, a 3B-parameter song model from September 2026 that turns lyrics plus a style description into a track with singing and backing instruments. Its distinctive step is a symbolic plan: before any sound exists, the model sketches melody and harmony in ABC notation, which you can read, alter by hand or through an LLM agent, and pass back in. Rendering runs in stages inside one Mixture-of-Transformers network, with token prediction for structure, flow matching for the sound, and a VAE giving 48 kHz stereo. The clone installs with pip as a package imported as yue2, with stage-by-stage control, examples and an agent skill for songwriting and covers; a separate SheetSage2 transcriber turns recordings into scores, and ComfyUI has native nodes. It needs Linux, Python 3.12 and a 24 GB NVIDIA card with bfloat16 support. Code is Apache-2.0, but weights carry a non-commercial Creative Commons license with a creator clause: individuals may monetize songs, companies need a paid license. Best for musicians and researchers with a large GPU; YuE v1 is kept on a branch.