EchoMimic

Ant Group's Alipay terminal technology team released EchoMimic as research code for turning one still portrait plus a voice recording into a talking-head video. The pipeline is diffusion based: a reference UNet holds the identity of the source photo, a denoising UNet with a temporal motion module produces the frames, Whisper tiny encodes the audio, and a face locator injects facial landmarks, so the face can follow sound alone, a landmark sequence, or sound plus selected landmarks. Accelerated checkpoints cut a 240-frame clip from roughly seven minutes to under a minute on a V100. Weights download from Hugging Face or ModelScope, and the repo ships command-line inference scripts and a Gradio web app, with a community ComfyUI node available. Expect Linux, CUDA 11.7 or newer, and an NVIDIA card; the authors tested on GPUs ranging from 16 GB to 80 GB. Code is Apache-2.0, but the authors frame the release as academic research and the model card names no weight license, so clear commercial use first. Successors EchoMimicV2 and V3 live in separate repositories. Suits researchers and avatar or dubbing prototypers.