CogVideo 1.5

The CogVideoX line from Zhipu AI and Tsinghua generates short clips from text prompts or a starting image, and version 1.5, from November 2024, is the newest pair: a 5B text-to-video model rendering at 1360 by 768 and a 5B image-to-video model that accepts flexible frame sizes within set bounds. Both output 5 or 10 second videos at 16 frames per second from English prompts capped near 224 tokens. The repo bundles inference scripts for Hugging Face diffusers and the SwissArmyTransformer (SAT) stack, a Gradio demo, LoRA fine-tuning code, DDIM inversion, and a helper that uses an LLM to expand terse prompts into the long descriptions it was trained on. Memory needs are steep: the SAT path wants about 76 GB, while diffusers with CPU offloading and INT8 quantization can squeeze into roughly 7 to 10 GB, and even one A100 needs about 1000 seconds per 5 second clip, with quantized runs slower. Code is Apache 2.0, but the 1.5 weights use the CogVideoX License: research is free, and commercial use is free after registration up to one million monthly visits, past which a further license is required. Suited to researchers with big GPUs.