Sesame CSM

Conversational speech model from Sesame that generates Mimi audio codes from text and audio context.

Open SourceSelf HostedOffline CapableGPU Required
0.0 (0)

About

Sesame AI Labs, the company behind the viral Maya voice demo, open-sourced CSM-1B in March 2025 as the base model of its Conversational Speech Model line. Rather than plain text to speech, CSM generates Mimi RVQ audio codes from combined text and audio inputs, using a Llama-style transformer backbone paired with a smaller audio decoder, which lets it produce dialogue speech that responds naturally to the preceding conversation. The weights sit on Hugging Face as sesame/csm-1b under Apache-2.0, and the model is natively integrated into Hugging Face Transformers from version 4.52 onward. Running the reference code requires a CUDA-capable GPU, Python 3.10, and access to the gated Llama-3.2-1B download; Windows users swap in triton-windows for the standard triton dependency. The base release is not tuned to any specific voice and has limited non-English ability, and the usage terms prohibit impersonation and deception. With 14.7k GitHub stars it is one of the most watched open speech releases, used in research on context-aware dialogue synthesis and in self-hosted voice agents.

Should you use Sesame CSM?

Pick it when

Pick Sesame CSM when you are building or researching a voice agent whose replies should sound aware of the preceding conversation, conditioning on both text and prior audio, and you can run CUDA with Transformers 4.52 or the reference code.

Look elsewhere when

Skip it if you need a polished voice out of the box, non-English speech, or a setup without gated Llama-3.2-1B access; the base model is not tuned to any speaker. Orpheus ships named voices, and Dia2 streams dialogue.

Alternatives to Sesame CSM

  • Orpheus TTS

    Ships eight named voices plus emotion tags and vLLM streaming, ready for agents without first fine-tuning a base voice.

  • Dia2

    Also conditions on prior audio for consistent multi-turn voices and starts speaking from the first words, but it is research-grade and output varies between generations.

  • Dia TTS

    Writes a whole multi-speaker dialogue from a transcript with nonverbal sounds, easier for scripted conversations than CSM's context-driven turns.

  • Moshi

    Full-duplex speech-to-speech model that listens while talking, replacing the LLM plus TTS split entirely but needing about 24 GB of VRAM.

Reviews (0)

Leave a Review

No reviews yet. Be the first to review!

Details

Price
Free
Platform
Local/Desktop
Difficulty
Intermediate (3/5)
License
Apache-2.0
Added
Jul 29, 2026

Tags