Faster Whisper transcription with CTranslate2

Faster Whisper is a reimplementation of OpenAI's Whisper model built on top of CTranslate2, the fast inference engine for Transformer models. This project aims to deliver high-accuracy transcription with significantly improved speed and lower memory usage. By leveraging CTranslate2, Faster Whisper achieves up to four times speed improvements over the original OpenAI Whisper for the same level of accuracy, and this efficiency can be further enhanced through 8-bit quantization on both CPU and GPU. The result is a versatile transcription solution that scales from laptop CPUs to high-end GPUs, delivering rapid, publication-ready transcripts with rich features.

Overview and motivation

• Faster Whisper reimagines Whisper’s architecture around a performant inference core (CTranslate2), preserving the quality of the underlying models while optimizing runtime.
• The approach emphasizes memory efficiency, enabling larger models to run on devices with limited resources and supporting batch processing to maximize throughput.
• Quantization options (notably 8-bit) unlock additional speedups on both CPU and GPU without sacrificing substantial transcription accuracy.

A quick tour of the landscape

• Benchmarking against a variety of Whisper implementations demonstrates the speed advantage of Faster Whisper under realistic conditions.
• The project provides practical guidance for deploying on both CPU and GPU, including Docker-based setups and pip-installed workflows.
• The workflow is designed to be approachable for researchers and developers who want to experiment with, convert, or load converted Whisper models into a CTranslate2-backed runtime.

Benchmark snapshot: how speed shapes the experience The project collects time and memory metrics for several Whisper implementations on a representative task: transcribing approximately 13 minutes of audio. The results reveal meaningful differences in latency and resource usage across platforms and configurations.

Large-v2 model on GPU

• OpenAI Whisper, FP16, beam size 5: around 2 minutes 23 seconds.
• Whisper.cpp (Flash Attention), FP16, beam size 5: about 1 minute 5 seconds.
• Transformers (SDPA), FP16, beam size 5: roughly 1 minute 52 seconds.
• Faster Whisper, FP16, beam size 5: approximately 1 minute 3 seconds.
• Faster Whisper with batch_size=8, FP16, beam size 5: around 17 seconds (note the substantial throughput gain with batching).
• Faster Whisper, INT8, beam size 5: about 59 seconds.
• Faster Whisper with batch_size=8, INT8, beam size 5: around 16 seconds.
• VRAM usage varies by configuration but remains within the practical range for modern GPUs.

Distil-Whisper large-v3 model on GPU

• Transformers (SDPA), batch_size 16, FP16, beam size 5: around 46 minutes 12 seconds, with a Word Error Rate (WER) of about 14.801 on YouTube Commons data.
• Faster Whisper, batch_size 16, FP16, beam size 5: about 25 minutes 50 seconds, achieving a WER of 13.527 (a noticeable improvement).
• A note: GPU benchmarks were executed on a CUDA 12.4 environment using an NVIDIA RTX 3070 Ti 8GB, illustrating the practical performance envelope for consumer-grade GPUs.

Small model on CPU

• OpenAI Whisper, FP32, beam size 5: approximately 6 minutes 58 seconds, RAM usage around 2335 MB.
• Whisper.cpp, FP32, beam size 5: about 2 minutes 5 seconds, RAM usage around 1049 MB.
• Whisper.cpp with OpenVINO: around 1 minute 45 seconds, RAM usage ~1642 MB.
• Faster Whisper, FP32, beam size 5: about 2 minutes 37 seconds, RAM usage around 2257 MB.
• Faster Whisper with batch_size=8, FP32, beam size 5: about 1 minute 6 seconds, RAM usage ~4230 MB.
• Faster Whisper, INT8, beam size 5: around 1 minute 42 seconds, RAM usage ~1477 MB.
• Faster Whisper with batch_size=8, INT8, beam size 5: about 51 seconds, RAM usage ~3608 MB.
• These CPU results are typically achieved with an 8-thread setup on modern desktop CPUs (for example, an Intel Core i7-12700K).

What they mean for you

• If you’re targeting GPU, Faster Whisper offers robust speedups, especially in FP16 with moderate memory requirements, and even more dramatic gains when batching is used.
• For CPU-only deployments, Faster Whisper remains viable, providing a sweet spot between speed and memory, with quantization helping to push latency down further.
• The data illustrates that the same model family can deliver very different performance profiles depending on the runtime (CUDA, CPU, OpenVINO) and configuration (beam size, batch size, precision).

Requirements and installation: what you need to know

• Python: Faster Whisper requires Python 3.9 or greater.
• FFmpeg: Unlike some Whisper variants, Faster Whisper does not require a separate FFmpeg installation on the system. Instead, audio decoding happens through PyAV, which bundles the necessary FFmpeg libraries within its package.
• GPU considerations: If you plan to run on NVIDIA GPUs, you’ll need the CUDA ecosystem, including cuBLAS for CUDA 12 and cuDNN 9 for CUDA 12. A critical compatibility note is that current cTranslate2 releases target CUDA 12 and cuDNN 9. If you’re on CUDA 11 or cuDNN 8, you’ll need to downgrade cTranslate2 to 3.24.0 for CUDA 11/cuDNN 8, or to 4.4.0 for CUDA 12/cuDNN 8. The installation can be performed via pip with a forced reinstall of the desired cTranslate2 version:
• pip install --force-reinstall ctranslate2==4.4.0 (for CUDA 12 with cuDNN 9)
• or a version aligned with your CUDA/cuDNN stack.
• Alternative installation paths exist:
• Docker-based approach using official NVIDIA CUDA images (for example, nvidia/cuda:12.3.2-cudnn9-runtime-ubuntu22.04).
• Pip on Linux: follow steps to install NVIDIA libraries via pip (nvidia-cublas-cu12 and nvidia-cudnn-cu12).
• Purfview’s whisper-standalone-win library packages: Windows and Linux libraries in a single archive.
• The installation is supported via PyPI:
• pip install faster-whisper
• You can also install from the master branch or a specific commit:
• pip install --force-reinstall "faster-whisper @ https://github.com/SYSTRAN/faster-whisper/archive/refs/heads/master.tar.gz"
• pip install --force-reinstall "faster-whisper @ https://github.com/SYSTRAN/faster-whisper/archive/a4f1cc8f11433e454c3934442b5e1a4ed5e865c3.tar.gz"

Usage: getting transcription done

• Core API
• from faster_whisper import WhisperModel
• model_size = "large-v3"
• model = WhisperModel(modelsize, device="cuda", computetype="float16")
• segments, info = model.transcribe("audio.mp3", beam_size=5)
• print("Detected language '%s' with probability %f" % (info.language, info.language_probability))
• for segment in segments: print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))
• Important note: segments is a generator. Transcription runs when you iterate over it or when you coerce to a list.
• segments, _ = model.transcribe("audio.mp3")
• segments = list(segments) # transcription runs here

Batched transcription: a throughput-focused workflow

• The BatchedInferencePipeline provides a drop-in replacement for WhisperModel.transcribe, enabling higher throughput by batching.
• Example:
• from faster_whisper import WhisperModel, BatchedInferencePipeline
• model = WhisperModel("turbo", device="cuda", compute_type="float16")
• batched_model = BatchedInferencePipeline(model=model)
• segments, info = batchedmodel.transcribe("audio.mp3", batchsize=16)
• for segment in segments: print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))

Faster Distil-Whisper: compatibility and guidance

• Distil-Whisper checkpoints are compatible with Faster Whisper. The latest distil-large-v3 checkpoint (from HuggingFace) works out of the box with the Faster Whisper transcription workflow.
• Example usage:
• from faster_whisper import WhisperModel
• model_size = "distil-large-v3"
• model = WhisperModel(modelsize, device="cuda", computetype="float16")
• segments, info = model.transcribe("audio.mp3", beamsize=5, language="en", conditiononprevioustext=False)
• for segment in segments: print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))

Word-level timestamps and VAD filtering

• Word-level timestamps
• segments, _ = model.transcribe("audio.mp3", word_timestamps=True)
• for segment in segments: for word in segment.words: print("[%.2fs -> %.2fs] %s" % (word.start, word.end, word.word))
• VAD (Voice Activity Detection) filtering
• segments, _ = model.transcribe("audio.mp3", vad_filter=True)
• The default behavior removes long silences (silence longer than 2 seconds). You can customize with vad_parameters, for example:
• segments, _ = model.transcribe("audio.mp3", vadfilter=True, vadparameters=dict(minsilenceduration_ms=500))
• Logging
• You can configure Python logging to capture detailed runtime information:
• import logging
• logging.basicConfig()
• logging.getLogger("faster_whisper").setLevel(logging.DEBUG)

Going further: deeper options

• The WhisperModel class implements a broad set of transcription options. See the source for the full parameter surface and the available knobs to tune speed, memory, latency, and accuracy.
• For more advanced workflows, you can explore the WhisperModel API and its transcription options in the source implementation.

Community integrations: an ecosystem of open-source projects Faster Whisper has inspired and integrated with a broad array of community projects. A non-exhaustive list includes:

• speaches: an OpenAI-compatible server for faster-whisper, deployable with Docker and supporting streaming and live transcription.
• WhisperX: a Python library offering speaker diarization and precise word-level timestamps with wav2vec2 alignment.
• whisper-ctranslate2: a command-line client based on faster-whisper, compatible with the original OpenAI Whisper client.
• whisper-diarize: a speaker-diariation tool built on faster-whisper and NVIDIA NeMo.
• whisper-standalone-win: standalone CLI executables of faster-whisper for Windows, Linux, and macOS.
• asr-sd-pipeline: a scalable, modular end-to-end multi-speaker transcription solution implemented with AzureML pipelines.
• Open-Lyrics: a Python library that transcribes voice files with faster-whisper and translates/polishes results into .lrc files.
• wscribe: a flexible transcript tool supporting faster-whisper with exportable word-level transcripts.
• aTrain: a GUI implementation of faster-whisper for transcription and diarization on Windows and Linux.
• Whisper-Streaming: real-time Whisper-like streaming with faster-whisper as the backend, featuring a self-adaptive latency policy.
• WhisperLive: nearly-live Whisper transcription using faster-whisper as the backend.
• Faster-Whisper-Transcriber: a simple, user-friendly transcriber utility.
• Open-dubbing: AI dubbing system using machine learning to translate and synchronize dialogue across languages.
• Whisper-FastAPI: a lightweight API backend compatible with Whisper formats for easy integration.

Model conversion: bringing Whisper models into the CTranslate2 world

• Loading a model by size, such as WhisperModel("large-v3"), triggers an automatic download of the corresponding CTranslate2 model from Hugging Face Hub (Systran’s repo).
• Conversion tooling enables you to convert Whisper models from the Transformers ecosystem to the CTranslate2 format, preserving weights and enabling FP16 quantization.
• Example command to convert the original large-v3 Whisper model to a CTranslate2-friendly format:
• pip install transformers[torch] ct2-transformers-converter
• --model openai/whisper-large-v3 --output_dir whisper-large-v3-ct2
• --copyfiles tokenizer.json preprocessorconfig.json --quantization float16
• The conversion CLI is flexible:
• You can point to a Hub model or a local directory.
• If you skip tokenizer.json when copying files, the tokenizer configuration can be downloaded automatically later.
• The conversion API is exposed via ct2.converters.TransformersConverter for advanced users.
• Loading a converted model is straightforward:
• python
• from faster_whisper import WhisperModel
• model = faster_whisper.WhisperModel("whisper-large-v3-ct2")
• Or load via the Hub:
• model = faster_whisper.WhisperModel("username/whisper-large-v3-ct2")

Load a converted model: practical examples

• Local directory
• model = faster_whisper.WhisperModel("whisper-large-v3-ct2")
• Hub-hosted model
• model = faster_whisper.WhisperModel("username/whisper-large-v3-ct2")
• The workflow is designed to seamlessly switch between locally converted models and remote Hub-hosted variants, enabling flexible experimentation and deployment.

Performance comparisons: fair and thoughtful benchmarking

• When comparing Faster Whisper to other Whisper implementations, ensure consistent settings:
• Use the same transcription options (especially the same beam size).
• Be mindful that different implementations may default to different beam sizes, which can dramatically impact speed and WER.
• On CPU, ensure you’re using the same number of threads. If needed, set OMPNUMTHREADS to a specific value, for example:
• OMPNUMTHREADS=4 python3 my_script.py
• The goal is to compare apples to apples: equivalent model size, identical audio input, and equivalent post-processing steps.

Practical notes for deployment

• The framework supports GPU and CPU deployments with a variety of quantization strategies, enabling you to tailor latency, throughput, and memory usage to your constraints.
• The PyAV-based audio decoding avoids a hard dependency on FFmpeg at the system level, simplifying installation and reducing potential environment conflicts.
• The documentation emphasizes real-world usage patterns: batched inference for throughput, word-level timestamps for downstream alignment tasks, VAD for clean audio segmentation, and configurable logging for easier troubleshooting.

Visual and code references: a glance through the input materials

• The project provides a succinct set of commands and usage patterns that show how to:
• Install and configure the runtime for CUDA-enabled GPUs.
• Run batched and non-batched transcription pipelines.
• Enable word-level timestamps and VAD filtering to enhance downstream processing.
• Convert and load models from the Transformers ecosystem into the CTranslate2-based runtime.
• The included examples illustrate practical workflows, from basic transcription to advanced features like diarization-aware pipelines and streaming scenarios.

Conclusion: why Faster Whisper matters

• Faster Whisper emerges as a practical, high-performance transcription engine that sits at the intersection of accuracy, speed, and memory efficiency.
• By leveraging the speed of CTranslate2 and offering a rich feature set (word-level timestamps, VAD, batched inference, and Distil-Whisper compatibility), it enables developers to deploy Whisper-powered transcription in production environments with confidence.
• The ecosystem around Faster Whisper—ranging from community integrations to model conversion tools—helps users move from theory to scalable, production-grade transcription solutions with relative ease.

Images and visuals

• Inline deployment and status badges, including the CI badge and PyPI version badge, are included at the top of this post to reflect the project’s ongoing development and distribution status. The visuals from the input are incorporated to illustrate the project’s branding and ecosystem.

If you’re exploring Whisper-based transcription with speed and efficiency in mind, Faster Whisper offers a compelling path forward. It marries the accuracy of the Whisper family with the performance discipline of CTranslate2, delivering a flexible, scalable solution for researchers, developers, and production environments alike. Whether you’re benchmarking models on GPUs, streaming live audio with low latency, or converting your own Whisper models for optimized inference, the Faster Whisper toolkit provides the building blocks and practical guidance to get you there.