MMAudio

Silent AI-generated video gets its soundtrack from MMAudio, which watches the frames and reads an optional text prompt to synthesize synchronized sound effects and ambience, timing footsteps, impacts, and environmental audio to on-screen motion. The CVPR 2025 work from UIUC and Sony AI trains a single multimodal transformer jointly on audio-video-text and audio-text data, which is why the same checkpoint also works as a plain text-to-audio generator. Practicality is the draw: inference fits in about 6GB of GPU memory in 16-bit mode, install is a git clone plus pip, and the default large_44k_v2 checkpoint downloads automatically from Hugging Face, with a Gradio interface, Colab notebook, and hosted demos on Replicate and Hugging Face Spaces. Community wrappers have made it the usual foley stage bolted onto open video generators. Licensing splits: code is MIT while weights are CC-BY-NC-4.0, so commercial deployments need care. Training and evaluation scripts ship as well for anyone retraining on licensed data.