Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 0 additions & 70 deletions Docker.md

This file was deleted.

4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,8 +103,8 @@ PocketTTS language selection is a model-load option. When the model path points

## Docker

Docker CPU and CUDA images are available for both CLI and server use. See [Docker.md](Docker.md) for build commands and working Docker examples.

Docker CUDA and CPU images are available for both CLI and server use. See [docker.md](docs/docker.md) for
available images, build commands and working Docker examples.

## Model Manager and GGUF Downloads

Expand Down
91 changes: 67 additions & 24 deletions docs/docker.md
Original file line number Diff line number Diff line change
@@ -1,68 +1,111 @@
# Running audio.cpp in Docker

## Table of Contents

- [Prerequisites](#prerequisites)
- [Image Variants](#image-variants)
- [Published Images](#published-images)
- [Build Images locally](#build-images-locally)
- [Usage](#usage)
- [Examples](#examples)

## Prerequisites

- Docker must be installed and running on your system
- Docker must be installed and running on your system.
- For CUDA:
- The [NVIDIA container toolkit](https://github.com/NVIDIA/nvidia-container-toolkit) must be installed
- The [NVIDIA container toolkit](https://github.com/NVIDIA/nvidia-container-toolkit) must be installed.

## Image Variants

The following image variants are available:

- **full**: Provides the main tools **cli** and **server** and test binaries in one image. When running the container, the first argument selects the tool to execute.

The following backends are supported:
- **cuda12**
- **cuda13**
- **cpu**

The following architectures are supported:
- **amd64**
- **arm64**

## Images
## Published Images

The following image flavors are available for this project:
- **full**: This image provides the main tools ***cli*** and ***server***
and test binaries in one image. When running the container the first argument
selects the tool to execute.
Docker images are published daily when new commits are available. The images are provided
as multiarch images (amd64/arm64).

## Build the image
Pull the latest images using these tags:
- **cuda12**: `ghcr.io/0xshug0/audio.cpp:full-cuda12`
- **cuda13**: `ghcr.io/0xshug0/audio.cpp:full-cuda13`
- **cpu**: `ghcr.io/0xshug0/audio.cpp:full-cpu`

Images for a specific day/commit can be found in the
[versions](https://github.com/0xShug0/audio.cpp/pkgs/container/audio.cpp/versions?filters%5Bversion_type%5D=tagged)
history.
The format is: `full-<backend>-<date>-<shortsha>`, e.g. `full-cuda12-20260725-db7d2c4`


## Build Images locally

If you would like to build the images locally, you can use the available
Dockerfiles in `.devops`.

### CUDA

Build with default CUDA 12.x version (see *.devops/cuda.Dockerfile*):
Build with the default CUDA 12.x version. See `.devops/cuda.Dockerfile`.

```bash
docker build -f .devops/cuda.Dockerfile -t local/audiocpp:full-cuda .
docker build -f .devops/cuda.Dockerfile -t local/audio.cpp:full-cuda12 .
```

Build with a specific CUDA version, e.g. 13.2.0:
Build with a specific CUDA version, for example 13.3.0:

```bash
docker build -f .devops/cuda.Dockerfile -t local/audiocpp:full-cuda --build-arg CUDA_VERSION=13.2.0 .
docker build -f .devops/cuda.Dockerfile -t local/audio.cpp:full-cuda13 --build-arg CUDA_VERSION=13.3.0 .
```

### CPU

```bash
docker build -f .devops/cpu.Dockerfile -t local/audiocpp:full-cpu .
docker build -f .devops/cpu.Dockerfile -t local/audio.cpp:full-cpu .
```

## Usage

The model directory `<models-dir>` must be mounted into the container.
An additional `<output-dir>` should be mounted for TTS tasks.
An additional `<output-dir>` should be mounted for TTS tasks.

### CUDA

```bash
docker run --rm --gpus all -v "<models-dir>:/models:ro" local/audiocpp:full-cuda <cli|server> <...>
docker run --rm --gpus all -v "<models-dir>:/models:ro" ghcr.io/0xshug0/audio.cpp:full-cuda12 <cli|server> --model /models/<model> <...>
```

### CPU

```bash
docker run --rm -v "<models-dir>:/models:ro" local/audiocpp:full-cpu <cli|server> <...>
docker run --rm -v "<models-dir>:/models:ro" ghcr.io/0xshug0/audio.cpp:full-cpu <cli|server> --model /models/<model> <...>
```

See the fully working [examples](#examples) below.
See the fully working [examples](#examples) below.

## Examples

Examples for Docker (CUDA and CPU) are available in `examples/docker`.
Examples for Docker, including CUDA and CPU, are available in `examples/docker`.

### Pocket TTS via docker run
### CLI

Generate speech via audio.cpp cli running in Docker.
The **[examples](../examples/docker/cli/EXAMPLE.md)** in `examples/docker/cli`
demonstrate how to run the audio.cpp CLI with `docker run`. The examples include:

See [EXAMPLE.md](../examples/docker/cli/EXAMPLE.md) in `examples/docker/cli`.
- **PocketTTS:** Text-to-Speech
- **Qwen3-TTS:** Text-to-Speech with Voice Cloning

### Pocket TTS via Docker Compose
### Server

Generate speech via audio.cpp server running in Docker (Compose).
The **[examples](../examples/docker/server/EXAMPLE.md)** in `examples/docker/server`
demonstrate how to run the audio.cpp server with `docker compose`. The examples include:

See [EXAMPLE.md](../examples/docker/server/EXAMPLE.md) in `examples/docker/server`.
- **PocketTTS:** Text-to-Speech
- **Qwen3-TTS:** Text-to-Speech with Voice Cloning
65 changes: 45 additions & 20 deletions examples/docker/cli/EXAMPLE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,47 +2,72 @@

Run the audio.cpp CLI tool inside a Docker container for text-to-speech.

## Setup
## PocketTTS

**1. Download the PocketTTS model**
### 1. Download the PocketTTS model

Get the English model from [kyutai/pocket-tts](https://huggingface.co/kyutai/pocket-tts/) on Hugging Face.
Place the files from `languages/english/` into `../models/pocket-tts/languages/english/`:
Get the English q8 model from
[audio-cpp/audio.cpp-gguf](https://huggingface.co/audio-cpp/audio.cpp-gguf/tree/main/PocketTTS-GGUF)
on Hugging Face.
Place the files from `english/` into `../models/PocketTTS-GGUF/english/`:

The directory should look like:
```
../models/pocket-tts/languages/english/
├── model.safetensors
├── tokenizer.model
../models/PocketTTS-GGUF/english/
├── pocket-tts-english-q8_0.gguf
└── embeddings/
├── alba.safetensors
└── ...
```

**2. Build the image**
### 2. Run

Run from repository root:

CPU:
Run one of:

```bash
docker build -f .devops/cpu.Dockerfile -t local/audiocpp:full-cpu .
./pocket-tts-cuda12.sh
./pocket-tts-cuda13.sh
./pocket-tts-cpu.sh
```

GPU (CUDA):
See the scripts for details. The scripts reference the published docker image
and save the generated speech to `output/speech.wav`.

```bash
docker build -f .devops/cuda.Dockerfile -t local/audiocpp:full-cuda .
## Qwen3-TTS (with voice cloning)

### 1. Download the Qwen3-TTS model

Get the Qwen3-TTS-12Hz-1.7B-Base-GGUF q8 model from
[audio-cpp/audio.cpp-gguf](https://huggingface.co/audio-cpp/audio.cpp-gguf/tree/main/Qwen3-TTS-12Hz-1.7B-Base-GGUF)
on Hugging Face.
Place the file into `../models/Qwen3-TTS-12Hz-1.7B-Base-GGUF/`:

The directory should look like:
```
../models/Qwen3-TTS-12Hz-1.7B-Base-GGUF/
└── qwen3-tts-12hz-1.7b-base-q8_0_v2.gguf
```

## Usage
### 2. Add reference audio and transcription

Put a `ref_audio.wav` and `ref_text.txt` in `../references/`.

The directory should look like:
```
../references/
├── ref_audio.wav
└── ref_text.txt
```

### 3. Run

Run one of:

```bash
./cpu-tts.sh
./cuda-tts.sh
./qwen3-tts-cuda12.sh
./qwen3-tts-cuda13.sh
./qwen3-tts-cpu.sh
```

Speech is saved to `output/speech.wav`.

See the scripts for details. The scripts reference the published docker image
and save the generated speech based on the provided reference audio in `output/speech.wav`.
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@ docker run --rm \
--user "$(id -u):$(id -g)" \
-v "$MODEL_DIR:/models:ro" \
-v "$OUTPUT_DIR:/output" \
local/audiocpp:full-cpu \
ghcr.io/0xshug0/audio.cpp:full-cpu \
cli \
--task tts \
--family pocket_tts \
--model /models/pocket-tts \
--model /models/PocketTTS-GGUF/english/pocket-tts-english-q8_0.gguf \
--backend cpu \
--text "You are successfully running a text-to-speech model using audio.cpp, a pure C++ inference engine for audio models." \
--voice-id alba \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@ docker run --rm \
--user "$(id -u):$(id -g)" \
-v "$MODEL_DIR:/models:ro" \
-v "$OUTPUT_DIR:/output" \
local/audiocpp:full-cuda \
ghcr.io/0xshug0/audio.cpp:full-cuda12 \
cli \
--task tts \
--family pocket_tts \
--model /models/pocket-tts \
--model /models/PocketTTS-GGUF/english/pocket-tts-english-q8_0.gguf \
--backend cuda \
--device 0 \
--text "You are successfully running a text-to-speech model using audio.cpp, a pure C++ inference engine for audio models." \
Expand Down
24 changes: 24 additions & 0 deletions examples/docker/cli/pocket-tts-cuda13.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/usr/bin/env bash
set -e

OUTPUT_DIR="$(cd "$(dirname "$0")" && pwd)/output"
mkdir -p "$OUTPUT_DIR"
MODEL_DIR="$(cd "$(dirname "$0")/../models" && pwd)"

docker run --rm \
--gpus all \
--user "$(id -u):$(id -g)" \
-v "$MODEL_DIR:/models:ro" \
-v "$OUTPUT_DIR:/output" \
ghcr.io/0xshug0/audio.cpp:full-cuda13 \
cli \
--task tts \
--family pocket_tts \
--model /models/PocketTTS-GGUF/english/pocket-tts-english-q8_0.gguf \
--backend cuda \
--device 0 \
--text "You are successfully running a text-to-speech model using audio.cpp, a pure C++ inference engine for audio models." \
--voice-id alba \
--out /output/speech.wav

echo "Saved to: $OUTPUT_DIR/speech.wav"
25 changes: 25 additions & 0 deletions examples/docker/cli/qwen3-tts-cpu.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/usr/bin/env bash
set -e

OUTPUT_DIR="$(cd "$(dirname "$0")" && pwd)/output"
mkdir -p "$OUTPUT_DIR"
MODEL_DIR="$(cd "$(dirname "$0")/../models" && pwd)"
REFERENCES_DIR="$(cd "$(dirname "$0")/../references" && pwd)"

docker run --rm \
--user "$(id -u):$(id -g)" \
-v "$MODEL_DIR:/models:ro" \
-v "$REFERENCES_DIR:/references:ro" \
-v "$OUTPUT_DIR:/output" \
ghcr.io/0xshug0/audio.cpp:full-cpu \
cli \
--task tts \
--family qwen3_tts \
--model /models/Qwen3-TTS-12Hz-1.7B-Base-GGUF/qwen3-tts-12hz-1.7b-base-q8_0_v2.gguf \
--backend cpu \
--text "You are successfully running a text-to-speech model using audio.cpp, a pure C++ inference engine for audio models." \
--voice-ref /references/ref_audio.wav \
--reference-text "$(cat "$REFERENCES_DIR/ref_text.txt")" \
--out /output/speech.wav

echo "Saved to: $OUTPUT_DIR/speech.wav"
Loading
Loading