WhisperX

Max Bain's pipeline, presented at INTERSPEECH 2023, fixes two weak spots in Whisper: loose timing and slow sequential decoding. A speech detector first cuts the audio into chunks, which are transcribed in batches through the faster-whisper (CTranslate2) backend, and the text is then force-aligned against a language-specific wav2vec2 phoneme model so each word receives accurate offsets. An optional third stage uses pyannote's community diarization model to tag words with speaker labels, which needs a Hugging Face token and acceptance of that model's terms. It installs through pip or uvx, giving a whisperx command (SRT with highlighted words, int8 CPU mode, speaker count hints) and a Python API exposing each stage. GPU use expects the CUDA 12.8 toolkit; large-v2 with beam search stays under 8 GB of VRAM, and smaller checkpoints or int8 shrink that further. Alignment models for English, French, German, Spanish and Italian come from torchaudio, with Hugging Face models covering many more languages. BSD-2-Clause licensing allows commercial use. Suited to subtitling, meeting transcripts and speech dataset preparation.