GLM-4-Voice

An early fully open end-to-end speech chat model, GLM-4-Voice from Zhipu AI holds a conversation in Chinese and English without a separate ASR or TTS pipeline. Three released components divide the work: a speech tokenizer that adds vector quantization to a Whisper encoder and compresses audio to 12.5 discrete tokens per second, the GLM-4-Voice-9B language model, extended from GLM-4-9B with pretraining on millions of hours of audio and interleaved speech-text data, and a flow-matching decoder derived from CosyVoice that can start synthesizing after as few as 10 tokens for low-latency streaming replies. Users can steer emotion, intonation, speaking rate, and dialect through natural language instructions. The repository provides a web demo with bfloat16 and Int4 launch options, and inference requires a CUDA GPU with substantial memory for the 9B model. Code is Apache-2.0 while the model weights carry their own usage agreement on Hugging Face. Released in December 2024 with an accompanying arXiv paper, it became a common baseline for the wave of open speech-to-speech models that followed.