AudioLDM

Published at ICML 2023 by Haohe Liu and colleagues at the University of Surrey, AudioLDM turns a text prompt into short clips of sound effects, ambience, music or rough speech by running diffusion in a compressed latent space conditioned on CLAP, a joint audio and text embedding that let the authors train on audio without captions. Beyond text-to-audio, the code can create a clip resembling an input recording, restyle a recording toward a text description with adjustable strength, and run super-resolution and inpainting from Python. It installs from GitHub with pip and provides an audioldm command, a small Gradio app for text prompts and six checkpoints in three sizes; the same models load through AudioLDMPipeline in Hugging Face Diffusers. An NVIDIA GPU is the practical route; a 2025 commit added Apple MPS. Output varies by seed, so it can generate several candidates and keep the best. The code and Hugging Face checkpoints are CC BY-NC-SA 4.0, while the Zenodo copies the CLI fetches are tagged CC BY 4.0, so clear commercial use with the authors. AudioLDM 2 lives in its own repository; this original remains a common baseline.