ToonCrafter

Traditional frame interpolation assumes small linear motions and breaks down on cartoons, where exaggerated movement and occlusion are the norm, so ToonCrafter treats inbetweening as a generative task: given two cartoon keyframes, a pretrained image-to-video diffusion prior adapted to the cartoon domain synthesizes up to 16 in-between frames at 512x320 resolution, taking around 24 seconds per clip on an A100 with 50 DDIM sampling steps. Optional sparse sketch guidance lets an animator steer the motion path with a few drawn strokes, and the same model supports reference-based sketch colorization. The work comes from CUHK and Tencent AI Lab and was published in the SIGGRAPH Asia 2024 journal track. Running it locally means a conda environment with Python 3.8.5 and roughly 24 GB of VRAM, which community optimizations bring closer to 10 GB, and ComfyUI nodes, Colab notebooks, and Windows ports exist. Code and weights are Apache-2.0 licensed with about 6,000 GitHub stars; the authors describe it as research exploration rather than a polished commercial product, without guaranteed results on every input.