Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# AGENTS.md

Notes for agents/contributors working with this repo on specific hardware, based on
measurements on a real GPU. Doc-only — nothing here changes code or defaults.

## NVIDIA Tesla T4 (Turing, sm_75)

Measured on a real Tesla T4 16GB (driver 550.163.01, torch 2.5.1+cu121), default
Chatterbox-Turbo engine, commit `915ae28`.

- **Keep `TTS_BF16` off (the default) on T4 / other Turing (sm_75) cards.**
`TTS_BF16=auto` enables bf16 whenever `torch.cuda.is_bf16_supported()` is `True`,
and T4 reports `True` for that check — but Turing has no bf16-capable tensor
cores, so bf16 falls back to a slow compute path instead of speeding things up.
Measured warm RTF (wall-clock / audio-length, 5 warm runs each, <0.5% run-to-run
variance): **0.335 fp32 (default) vs 0.535 bf16 (`TTS_BF16=on`/`auto`) — bf16 is
~1.6x slower.** bf16 does reduce peak VRAM (~3.9GB vs ~4.9GB), but on a 16GB
card that headroom isn't needed. The README's "~40% throughput on bf16-capable
GPUs" claim holds on Ampere+ (A100/A10/RTX 30xx+), not on Turing.
This repo doesn't expose an fp16 path, so float32 (the default) is the best
option on T4-class hardware.
- **The default Turbo engine ignores `exaggeration` and `cfg_weight`.** The
`/tts` API and `config.generation_defaults` expose these as if they were always
tunable, but Turbo logs `CFG, min_p and exaggeration are not supported by Turbo
version and will be ignored` at runtime. If you need exaggeration/CFG control,
switch to the Original or Multilingual engine instead of Turbo.
15 changes: 10 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -505,12 +505,13 @@ This is the most straightforward option and works on any machine without a compa
# Make sure your (venv) is active
pip install --upgrade pip
pip install -r requirements.txt
pip install --no-deps git+https://github.com/devnen/chatterbox-v2.git@master
pip install --no-deps git+https://github.com/devnen/chatterbox-v2.git@master s3tokenizer==0.3.0 onnx==1.16.0
pip install --no-deps --force-reinstall "protobuf>=4.25.0"
```

<details>
<summary><strong>💡 How This Works</strong></summary>
The `requirements.txt` file installs CPU PyTorch and all server dependencies. Chatterbox is installed separately with `--no-deps` to prevent pip from pulling in conflicting torch versions or triggering ONNX source builds.
The `requirements.txt` file installs CPU PyTorch and all server dependencies. Chatterbox is installed separately with `--no-deps` to prevent pip from pulling in conflicting torch versions or triggering ONNX source builds. The `s3tokenizer`/`onnx`/`protobuf` pins are required regardless of install type — `start.py`'s `install_chatterbox_no_deps()` step needs them, and omitting them causes `ModuleNotFoundError: No module named 's3tokenizer'` at server startup.
</details>

---
Expand All @@ -525,7 +526,8 @@ For users with NVIDIA GPUs. This provides the best performance for RTX 20/30/40
# Make sure your (venv) is active
pip install --upgrade pip
pip install -r requirements-nvidia.txt
pip install --no-deps git+https://github.com/devnen/chatterbox-v2.git@master
pip install --no-deps git+https://github.com/devnen/chatterbox-v2.git@master s3tokenizer==0.3.0 onnx==1.16.0
pip install --no-deps --force-reinstall "protobuf>=4.25.0"
```

**After installation, verify that PyTorch can see your GPU:**
Expand Down Expand Up @@ -568,7 +570,8 @@ pip install --upgrade pip
pip install -r requirements-nvidia-cu128.txt

# Step 2: Install chatterbox without dependencies (prevents PyTorch downgrade)
pip install --no-deps git+https://github.com/devnen/chatterbox-v2.git@master
pip install --no-deps git+https://github.com/devnen/chatterbox-v2.git@master s3tokenizer==0.3.0 onnx==1.16.0
pip install --no-deps --force-reinstall "protobuf>=4.25.0"
```

⚠️ **Critical:** The `--no-deps` flag is required to prevent PyTorch from being downgraded to a version that doesn't support Blackwell GPUs.
Expand Down Expand Up @@ -1135,9 +1138,11 @@ The primary endpoint for TTS generation is `/tts`. The OpenAI-compatible `/v1/au
```bash
curl -X POST http://localhost:8004/tts \
-H "Content-Type: application/json" \
-d '{"text":"The first chunk arrives quickly, the rest stream behind.","stream":true}' \
-d '{"text":"The first chunk arrives quickly, the rest stream behind.","predefined_voice_id":"Emily.wav","stream":true}' \
--output stream.wav
```
> **Note:** `predefined_voice_id` is required when `voice_mode` is left at its default (`"predefined"`) — the server returns `400 Missing 'predefined_voice_id' for 'predefined' voice mode.` without it. Use any filename from `voices/` (e.g. `Emily.wav`), or set `"voice_mode":"clone"` with `reference_audio_filename` instead.

# 🐳 Docker Installation

Run Chatterbox TTS Server easily using Docker. The recommended method uses Docker Compose, which is pre-configured for different GPU types.
Expand Down