Chatterbox TTS Server: OpenAI-Compatible TTS with Multi-Engine Web UI

Chatterbox TTS Server: OpenAI-Compatible API with Web UI, Large Text Handling & Built-in Voices

Introduction

The Chatterbox TTS Server is a self-hosted, open-source solution that brings the power of Resemble AI’s Chatterbox family behind an OpenAI-compatible API and a modern web user interface. It unifies three distinct engines—Original Chatterbox, Chatterbox Multilingual, and Chatterbox Turbo—into a single, easy-to-use server with a single API surface. Designed for accessibility and robustness, it supports voice cloning, large-text processing with intelligent chunking, audiobook generation, and consistent, repeatable synthesis through predefined voices and a generation seed. With GPU acceleration on NVIDIA, AMD, and Apple Silicon, plus fallbacks to CPU, the server is built for a wide range of hardware profiles and deployment needs.

Images from Input

• Chatterbox TTS Server Web UI - Dark Mode: static/screenshot-d.png
• Chatterbox TTS Server Web UI - Light Mode: static/screenshot-l.png

A Quick Overview: The Chatterbox Family in One Server

• Original Chatterbox: High-quality English voice output with emotion exaggeration control, backed by a 0.5B-parameter model and a 0.5M hours of curated training data.
• Chatterbox Multilingual: 23-language support, including Arabic, Chinese, Danish, Dutch, English, French, German, Hindi, Japanese, Korean, Portuguese, Russian, Spanish, Turkish, and more. Provides voice cloning and emotion control on the same 0.5B-parameter backbone.
• Chatterbox Turbo: A streamlined 350M-parameter model designed for throughput and real-time workflows. Turbo introduces native paralinguistic tags such as [laugh], [cough], and [chuckle], enabling more natural agent-style narration and expressive reads.

A Unified API and Web UI

• OpenAI-compatible API: The server responds to OpenAI-style prompts, enabling easy integration with existing OpenAI-compatible clients and tooling.
• Web UI: A modern interface for experimenting with inputs, loading presets, managing reference audio, tuning generation settings, and playing back results.
• Hot-swappable engines: The Web UI includes an engine selector that instantly switches among Original, Multilingual, and Turbo—no server restarts required. All UI and API requests route through the active engine.

Images and visuals

• The included dark- and light-mode screenshots illustrate a contemporary, responsive web interface designed for text input, voice selection, and playback controls. The visuals show a clean layout with an engine dropdown, presets, reference audio handling, and a waveform audio player for generated output.

Key Features and Capabilities

• Multi-engine support: Original Chatterbox, Chatterbox Multilingual, and Chatterbox Turbo are all accessible from a single UI/API surface and can be swapped live.
• Paralinguistic prompts (Turbo): Inline tags such as [laugh], [cough], and [chuckle] enable expressive narration directly in the text, expanding the expressive range of the synthesized voice.
• Large text handling: Intelligent chunking splits long inputs into sentence-based segments, processes them sequentially, and concatenates the audio seamlessly for audiobook-style generation.
• Voice cloning and predefined voices: Use a reference audio to clone a voice or select from ready-to-use predefined voices stored locally for consistent output without cloning setup.
• Seed-based consistency: A generation seed ensures replicable results across runs or across multiple chunks, especially useful in OpenAI-compatible pipelines.
• Audiobook generation: Designed to produce cohesive, continuous narration for full-length texts with consistent voice quality.
• Cross-platform acceleration: NVIDIA CUDA, AMD ROCm, and Apple MPS are supported with automatic CPU fallback if no accelerator is detected.
• Large-audio streaming: A streaming option (/tts with stream: true) returns WAV chunks with 20 ms crossfades for long-form content.
• Accessibility and configurability: A dedicated UI for editing and persisting configuration, generation defaults, and UI state to ensure repeatable workflows.

New and Notable Enhancements in v2.0.0

• DGX Spark / sm_121 support: CUDA 13.0, PyTorch 2.10, via docker-compose-cu130.yml, for advanced GPU clusters.
• AMD Strix Halo support: ROCm 7.2 integration via docker-compose-strixhalo.yml.
• Streaming endpoint: The /tts endpoint can stream WAV bytes in chunks for long-form content, while maintaining default non-streaming behavior for standard use cases.
• Voice conditioning cache: Repeated requests against the same reference voice bypass re-encoding, reducing latency in batch and API workflows.
• BF16 inference (TTSBF16): Optional BF16 acceleration with TTSBF16=on or =auto, delivering roughly 40% throughput gains on bf16-capable GPUs, while preserving default behavior if disabled.
• HTTPS/SSL support: Optional sslcertfile and sslkeyfile in config.yaml for direct HTTPS without an external reverse proxy.
• Security improvements: CWE-22 path traversal fixed on /tts and /v1/audio/speech, returning HTTP 400 for unsafe inputs.
• New endpoints: /api/unload to release GPU memory without restarting, and /v1/audio/voices for OpenAI-compatible voice listings.
• Dynamic language selector: The UI dynamically populates the language dropdown from SUPPORTED_LANGUAGES exposed by the multilingual engine.
• Chunker fixes: Ensures narrative text doesn’t get misinterpreted as bullet items due to stray dashes (#144).
• Portable Mode for Windows: A fully self-contained Windows installation that includes Python and dependencies; start by launching start.bat—no system Python installation required.
• Complete multilingual and Turbo integration: A cohesive experience across all three engines, with seamless switching and consistent UI behavior.

Portable Mode for Windows: Self-Contained, Moveable, and Convenient

• Portable Mode packages the entire application folder with an embedded Python runtime, so you can copy to a USB drive, zip and share, or move it anywhere on disk. Double-click start.bat to run.
• First-time setup includes a portable Python environment and model downloads; subsequent runs are fast and do not require internet access for the server to start.
• The models themselves are cached in a Hugging Face cache on first run; you can configure model_cache in config.yaml to pre-place models within the portable package for offline distribution.

Chatterbox Multilingual: Global Reach with Local Voice Quality

• 23 languages span a broad spectrum of dialects and scripts, enabling multilingual audiobook projects, voice agents, and international applications.
• The Multilingual model maintains the same 0.5B parameter footprint as Original, preserving emotion exaggeration control and zero-shot voice cloning capabilities.

Chatterbox Turbo: Speed, Efficiency, and Paralinguistic Expression

• Turbo trims the diffusion decoding from 10 steps to 1 step, delivering dramatically higher throughput.
• The 350M-parameter architecture preserves quality while enabling faster-than-real-time generation on capable GPUs.
• Native paralinguistic tags empower expressive agent narrations and more natural conversational agents.

Paralinguistic Tags and Narration Expressiveness

• Turbo supports embedded tags such as [laugh], [cough], and [chuckle] to convey reactions and emphasis within the generated speech.
• New presets in ui/presets.yaml demonstrate how these tags can drive agent-style scripting and expressive reads.

Voice Cloning, Predefined Voices, and Consistency

• Voice Cloning: Upload a reference audio to produce a close match to a target voice.
• Predefined Voices: A curated set of ready-to-use voices stored in the voices/ directory provide consistent output without cloning steps.
• Seed-based consistency: Pairing predefined voices or voice cloning with a fixed seed yields stable outputs across runs and across text chunks.

Installation, Setup, and Deployment Options

• Automated Launcher (Recommended): A one-command setup that detects hardware, creates a virtual environment, installs dependencies, downloads models, starts the server, and opens the Web UI. Includes upgrade and reinstall options.
• Portable Windows Mode: The Windows launcher enables a fully self-contained, movable installation with an embedded Python 3.10 runtime.
• Manual Installation: Several pathways for CPU, NVIDIA CUDA (12.1, 12.8, 13.0), ROCm (AMD), and Apple Silicon MPS, with detailed commands and caveats.
• Docker support: Docker and Docker Compose enable reproducible deployments across CPU, NVIDIA, ROCm, and other configurations. Separate compose files exist for cu128 (Blackwell), cu130 (DGX Spark), and ROCm-based setups; CPU-only deployment also has its own file.
• Model downloads: The server downloads chatterbox-tts models from Hugging Face on first run. A separate script is available to pre-download models into model_cache if desired.

System Prerequisites and Hardware Considerations

• Operating Systems: Windows 10/11 (64-bit) or Linux (Debian/Ubuntu recommended); macOS is supported via Apple Silicon with MPS.
• Python: Version 3.10 is required for pre-built wheels and compatibility with the pinned dependencies. Python 3.11+ is not supported in standard installations.
• GPUs:
• NVIDIA CUDA-enabled GPUs (12.1, 12.8, or 13.0 depending on the build and hardware) with compatible driver versions.
• AMD ROCm-enabled GPUs on Linux, with ROCm 6.1+ and compatible PyTorch builds.
• Apple Silicon GPUs (MPS) on macOS 12.3+.
• Other requirements: Sufficient disk space (10 GB+ for dependencies and model caches), internet access for initial model downloads, and container tools if using Docker.

API and Web UI: How to Use

• Core endpoint: /tts, which accepts a rich parameter set to control text, voice mode, predefined voice, reference audio, chunking behavior, streaming, generation controls (temperature, exaggeration, CFG weight, seed, speed factor, language), and output format (wav/mp3/opus).
• OpenAI-compatible endpoints: /v1/audio/speech for drop-in compatibility and /v1/audio/voices for listing available voices.
• Streaming: When stream is true, the server yields WAV data in chunks with 20 ms crossfades. Streaming is chunk-level rather than token-level.
• UI persistence: The UI stores last-used text, voice mode, file selections, and generation parameters in config.yaml, making it easy to resume sessions.
• Presets: The ui/presets.yaml file provides example configurations for rapid testing and demonstration of Turbo’s paralinguistic features.
• Reference and predefined audio management: The UI supports uploading reference audio files and managing predefined voices stored under the voices/ directory.

Security and Robustness

• Path traversal protections (CWE-22): Voice file parameters are sandboxed under designated directories, and any traversal attempts are rejected with HTTP 400.
• HTTPS/SSL: Optional TLS configuration enables direct HTTPS without requiring a reverse proxy.
• Safe file handling: Inputs are constrained to safe paths, reducing the risk of unauthorized filesystem access.

Performance Tuning and Best Practices

• BF16 inference: When enabled, training or inference runs in bf16 mode with autocast for improved throughput on capable hardware.
• Voice conditioning cache: Re-uses decoded representations of reference voices to avoid repeated re-encoding.
• Chunking controls: Adjust splittext and chunksize to balance between latency and memory usage. Larger chunks decrease the number of requests but require more VRAM per call.
• Streaming considerations: Use streaming for long-form content, but be mindful that short texts may not benefit latency-wise due to the chunk-level nature of the streaming approach.
• HTTPS in production: For direct HTTPS, configure sslcertfile and sslkeyfile in config.yaml to remove the need for a reverse proxy.

Running and Maintaining the Server

• Quick start with the Colab demo: A one-click Google Colab demo provides a live, browser-based test environment with voice cloning and audiobook generation, ideal for quick experimentation without local setup.
• Local startup: Use the Automated Launcher for Windows or the start.sh script on Linux/macOS. After the initial model downloads, subsequent starts are significantly faster.
• Upgrading: The automated launcher supports an upgrade flow that preserves hardware selection while updating code and dependencies. A full reinstall option is available to switch hardware configurations.
• Docker: For reproducible deployments, Docker Compose files provide GPU-aware configurations for NVIDIA, AMD, and CPU-only environments.

Usage Scenarios and Applications

• Audiobook production: The large-text chunking system and consistent voice output make it feasible to convert long-form texts into professional-quality audiobooks with minimal manual intervention.
• Global voice agents: With multilingual support and paralinguistic capabilities, the server is well-suited for international virtual assistants and customer-service agents.
• Prototyping and testing: The engine selector and OpenAI-compatible API enable rapid A/B testing of different voices and languages in the same application.
• Education and accessibility: The combination of high-quality English output and multilingual capabilities expands access to information through natural-sounding speech.

What’s Inside: A Developer- and User-Focused Architecture

• Core engine: The chatterbox-tts engine, built on the Resemble AI Chatterbox models, forms the foundation for all three engines, each offering distinct strengths.
• FastAPI server: A robust and scalable API layer built with FastAPI, providing a clean surface for programmatic integration and a Swagger-based docs experience at /docs.
• UI and UX: A modern, responsive web interface supports presets, audio playback, reference management, and a configurable UI state that persists across sessions.
• Model loading and caching: The system leverages Hugging Face Hub caches for model downloads and a local model_cache option for pre-downloaded components.
• Cross-platform acceleration: The server automatically detects available hardware and uses the appropriate backend (CUDA, MPS, ROCm) with a safe CPU fallback.
• Safety and expandability: The configuration is centralized in config.yaml with a clear separation between runtime behavior and UI state, ensuring predictable deployment and straightforward upgrades.

Contributing and Licensing

• This project builds on an open-source stack and welcomes contributions. The core model is derived from Resemble AI’s Chatterbox, and the server is released under MIT licensing terms.
• The project acknowledges its dependencies and inspiration from related open-source projects in the space and invites collaboration to extend multilingual capabilities, performance, and deployment options.

Takeaways: Why This Server Stands Out

• It delivers a one-stop TTS solution that combines multiple Chatterbox engines under a single, OpenAI-compatible API and a polished Web UI.
• The Turbo engine introduces practical paralinguistic capabilities that increase expressiveness without requiring external modeling steps.
• The system excels at long-form text, enabling audiobook workflows with consistent voices and automated chunking.
• It supports a wide hardware spectrum, from consumer GPUs to high-end accelerators, with thoughtful defaults and straightforward upgrade paths.
• Portable Windows mode and Docker-based deployments make it feasible to bring the server into varied environments with minimal friction.

API and Documentation Access

• Interactive API docs are available through the Swagger UI at /docs.
• The OpenAI-compatible surface includes /v1/audio/speech for drop-in compatibility and /v1/audio/voices for voice listings.
• The health and status endpoints, such as /api/ui/initial-data, provide bootstrap data and a comprehensive status snapshot of the server.

Final Thoughts

Chatterbox TTS Server synthesizes high-quality speech, rich multilingual support, and expressive retellings into a cohesive, developer-friendly package. Its emphasis on large-text handling, voice consistency, and flexible deployment options makes it suitable for independent projects, research, and production workflows. The ability to switch between Original, Multilingual, and Turbo engines on the fly, combined with paralinguistic tagging and audiobooks capabilities, positions this server as a versatile tool for anyone looking to generate natural-sounding speech at scale—whether for a personal project, a startup, or a library of multilingual digital assistants.

Endnote: Visualizes and Demos

• If you’re curious to see the UI in action, check the included dark-mode and light-mode screenshots to get a sense of the layout, controls, and playback features that users will interact with when generating speech through the OpenAI-compatible API or the web interface.