From d87329ccde25b50cab68a0abf0e5a119bb029040 Mon Sep 17 00:00:00 2001 From: moduvoice Date: Fri, 10 Jul 2026 18:40:12 +0700 Subject: [PATCH 1/2] docs: add Tesla T4 (Turing) hardware notes - Fix Manual Install Option 2/2b pip commands to include the s3tokenizer==0.3.0 and onnx==1.16.0 pins plus the protobuf upgrade, matching what start.py already installs automatically. Without these, the base ChatterboxTTS import fails with ModuleNotFoundError: No module named 's3tokenizer'. - Add predefined_voice_id to the /tts curl example (and a note explaining why it's required for the default predefined voice mode) since the example as written returns HTTP 400. - Add AGENTS.md documenting that TTS_BF16 should stay off (default) on T4/Turing (sm_75) GPUs, since is_bf16_supported() returns True there but bf16 is measured ~1.6x slower (RTF 0.535 vs 0.335) with no real hardware acceleration, and noting that the default Turbo engine ignores exaggeration/cfg_weight. --- AGENTS.md | 26 ++++++++++++++++++++++++++ README.md | 10 +++++++--- 2 files changed, 33 insertions(+), 3 deletions(-) create mode 100644 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..41b210b --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,26 @@ +# AGENTS.md + +Notes for agents/contributors working with this repo on specific hardware, based on +measurements on a real GPU. Doc-only — nothing here changes code or defaults. + +## NVIDIA Tesla T4 (Turing, sm_75) + +Measured on a real Tesla T4 16GB (driver 550.163.01, torch 2.5.1+cu121), default +Chatterbox-Turbo engine, commit `915ae28`. + +- **Keep `TTS_BF16` off (the default) on T4 / other Turing (sm_75) cards.** + `TTS_BF16=auto` enables bf16 whenever `torch.cuda.is_bf16_supported()` is `True`, + and T4 reports `True` for that check — but Turing has no bf16-capable tensor + cores, so bf16 falls back to a slow compute path instead of speeding things up. + Measured warm RTF (wall-clock / audio-length, 5 warm runs each, <0.5% run-to-run + variance): **0.335 fp32 (default) vs 0.535 bf16 (`TTS_BF16=on`/`auto`) — bf16 is + ~1.6x slower.** bf16 does reduce peak VRAM (~3.9GB vs ~4.9GB), but on a 16GB + card that headroom isn't needed. The README's "~40% throughput on bf16-capable + GPUs" claim holds on Ampere+ (A100/A10/RTX 30xx+), not on Turing. + This repo doesn't expose an fp16 path, so float32 (the default) is the best + option on T4-class hardware. +- **The default Turbo engine ignores `exaggeration` and `cfg_weight`.** The + `/tts` API and `config.generation_defaults` expose these as if they were always + tunable, but Turbo logs `CFG, min_p and exaggeration are not supported by Turbo + version and will be ignored` at runtime. If you need exaggeration/CFG control, + switch to the Original or Multilingual engine instead of Turbo. diff --git a/README.md b/README.md index f31e77b..9fb04ce 100644 --- a/README.md +++ b/README.md @@ -525,7 +525,8 @@ For users with NVIDIA GPUs. This provides the best performance for RTX 20/30/40 # Make sure your (venv) is active pip install --upgrade pip pip install -r requirements-nvidia.txt -pip install --no-deps git+https://github.com/devnen/chatterbox-v2.git@master +pip install --no-deps git+https://github.com/devnen/chatterbox-v2.git@master s3tokenizer==0.3.0 onnx==1.16.0 +pip install --no-deps --force-reinstall "protobuf>=4.25.0" ``` **After installation, verify that PyTorch can see your GPU:** @@ -568,7 +569,8 @@ pip install --upgrade pip pip install -r requirements-nvidia-cu128.txt # Step 2: Install chatterbox without dependencies (prevents PyTorch downgrade) -pip install --no-deps git+https://github.com/devnen/chatterbox-v2.git@master +pip install --no-deps git+https://github.com/devnen/chatterbox-v2.git@master s3tokenizer==0.3.0 onnx==1.16.0 +pip install --no-deps --force-reinstall "protobuf>=4.25.0" ``` ⚠️ **Critical:** The `--no-deps` flag is required to prevent PyTorch from being downgraded to a version that doesn't support Blackwell GPUs. @@ -1135,9 +1137,11 @@ The primary endpoint for TTS generation is `/tts`. The OpenAI-compatible `/v1/au ```bash curl -X POST http://localhost:8004/tts \ -H "Content-Type: application/json" \ - -d '{"text":"The first chunk arrives quickly, the rest stream behind.","stream":true}' \ + -d '{"text":"The first chunk arrives quickly, the rest stream behind.","predefined_voice_id":"Emily.wav","stream":true}' \ --output stream.wav ``` +> **Note:** `predefined_voice_id` is required when `voice_mode` is left at its default (`"predefined"`) — the server returns `400 Missing 'predefined_voice_id' for 'predefined' voice mode.` without it. Use any filename from `voices/` (e.g. `Emily.wav`), or set `"voice_mode":"clone"` with `reference_audio_filename` instead. + # 🐳 Docker Installation Run Chatterbox TTS Server easily using Docker. The recommended method uses Docker Compose, which is pre-configured for different GPU types. From 0aaa35d41069c6bf141d6294d12b422ca08358ea Mon Sep 17 00:00:00 2001 From: moduvoice Date: Fri, 10 Jul 2026 18:49:46 +0700 Subject: [PATCH 2/2] docs: fix Option 1 (CPU-only) install missing s3tokenizer/onnx/protobuf --- README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 9fb04ce..b5a6078 100644 --- a/README.md +++ b/README.md @@ -505,12 +505,13 @@ This is the most straightforward option and works on any machine without a compa # Make sure your (venv) is active pip install --upgrade pip pip install -r requirements.txt -pip install --no-deps git+https://github.com/devnen/chatterbox-v2.git@master +pip install --no-deps git+https://github.com/devnen/chatterbox-v2.git@master s3tokenizer==0.3.0 onnx==1.16.0 +pip install --no-deps --force-reinstall "protobuf>=4.25.0" ```
💡 How This Works -The `requirements.txt` file installs CPU PyTorch and all server dependencies. Chatterbox is installed separately with `--no-deps` to prevent pip from pulling in conflicting torch versions or triggering ONNX source builds. +The `requirements.txt` file installs CPU PyTorch and all server dependencies. Chatterbox is installed separately with `--no-deps` to prevent pip from pulling in conflicting torch versions or triggering ONNX source builds. The `s3tokenizer`/`onnx`/`protobuf` pins are required regardless of install type — `start.py`'s `install_chatterbox_no_deps()` step needs them, and omitting them causes `ModuleNotFoundError: No module named 's3tokenizer'` at server startup.
---