AI Audiobook Generator

Production-quality pipeline that converts technical markdown lectures into narrated audio using Kokoro-82M (ONNX) and Chatterbox-Turbo (Resemble AI), with loudness normalization, dynamic compression, and a side-by-side quality report for A/B comparison.

The reference document is ai_landscape_lecture.md — a ~45-minute technical talk on the AI capability stack. Running the pipeline produces two full audiobooks and a markdown comparison report under output/.

Highlights

Two TTS backends, identical text pipeline. Preprocessing, chunking, and post-processing are engine-agnostic; only the synthesis step differs.
Speech-friendly preprocessing. Headings become chapter markers; bullets become full-stop-separated sentences; code blocks are summarized, not read verbatim; URLs are replaced with a short reference phrase; Greek letters and math symbols are pronounced.
Chapter-aware chunking. 800–1200 word chunks, split at sentence boundaries when needed.
Long-form guard for Chatterbox-Turbo. The T3 component of Chatterbox early-terminates on inputs above ~200 words, so the runner sentence-aware-sub-chunks at ~150 words and concatenates the results.
Podcast-standard post-processing. -16 LUFS normalization, mild dynamic-range compression, leading/trailing silence trim, small inter- chapter gaps with crossfades.
Resumable. Per-chapter WAVs are cached. Re-running the pipeline skips cached output and only renders missing chapters.

Repository layout

.
├── ai_landscape_lecture.md        # source document
├── run.py                         # CLI entry point
├── requirements.txt
├── src/
│   ├── preprocess.py              # markdown -> speech-friendly text
│   ├── chunker.py                 # chapter-aware text chunking
│   ├── kokoro_runner.py           # Kokoro-82M ONNX pipeline
│   ├── chatterbox_runner.py       # Chatterbox-Turbo pipeline
│   ├── merger.py                  # concatenate + loudness + compression
│   ├── report.py                  # comparison_report.md generator
│   └── utils.py                   # shared dataclasses, logging, I/O
└── output/
    ├── kokoro/
    │   ├── kokoro.mp3             # full audiobook (Kokoro)
    │   ├── kokoro.wav             # intermediate WAV
    │   ├── voice_samples/         # per-voice A/B samples
    │   └── chapters/              # one WAV per chapter
    ├── chatterbox/
    │   ├── chatterbox.mp3         # full audiobook (Chatterbox-Turbo)
    │   ├── chatterbox.wav
    │   └── chapters/
    └── comparison_report.md

Setup

1. Requirements

Python 3.10+ (tested on 3.10 and 3.12)
Windows, macOS, or Linux
GPU optional — both engines work on CPU; GPU just speeds up Chatterbox
ffmpeg is required for MP3 export (auto-installed via static-ffmpeg)
espeak-ng is required by Kokoro-82M for phonemization
- Windows: winget install espeak-ng (or it ships with phonemizer wheels)
- macOS: brew install espeak-ng
- Linux: apt-get install espeak-ng

2. Create a virtual environment (optional but recommended)

python -m venv .venv
# Windows
.venv\Scripts\activate
# macOS / Linux
source .venv/bin/activate

3. Install dependencies

pip install -r requirements.txt

kokoro-onnx==0.3.0 and chatterbox-tts==0.1.7 are pinned because the two packages have mutually exclusive NumPy requirements (Kokoro 0.3.x needs NumPy 1.x; Chatterbox 0.1.7 also needs NumPy 1.x, but newer Kokoro releases require NumPy 2.x).

The first run will auto-download:

File	Size	Source
`kokoro-v0_19.onnx`	~310 MB	thewh1teagle/kokoro-onnx release
`voices.json`	~30 MB	thewh1teagle/kokoro-onnx release
Chatterbox-Turbo model files	~2 GB	Hugging Face (`ResembleAI/chatterbox-turbo`)

Model files are cached under ~/.cache/ai_audiobook/.

Usage

Run both engines end-to-end (recommended)

python run.py --engine both

This produces the full output tree and the comparison report.

Run a single engine

python run.py --engine kokoro
python run.py --engine chatterbox

Choose a Kokoro voice

python run.py --engine kokoro --voice af_sarah
python run.py --engine kokoro --voice am_adam
python run.py --engine kokoro --voice af_nicole

The default voice is am_michael. The spec's preference order is: am_michael > af_sarah > am_adam > af_nicole. Per-voice A/B samples are auto-generated under output/kokoro/voice_samples/ on the first run.

Adjust speech rate

python run.py --engine kokoro --speed 1.10
python run.py --engine both     --speed 0.95

Render a single chapter (smoke test / debugging)

python run.py --engine kokoro --chapter 0
python run.py --engine chatterbox --chapter 0

Tune Chatterbox-Turbo expressiveness

# Stable, measured delivery (default)
python run.py --engine chatterbox --exaggeration 0.30 --cfg-weight 0.40

# Slightly more expressive
python run.py --engine chatterbox --exaggeration 0.45 --cfg-weight 0.50

Voice cloning with Chatterbox-Turbo

python run.py --engine chatterbox --audio-prompt path/to/reference.wav

Skip parts of the pipeline

# Render per-chapter WAVs but skip the merge / MP3
python run.py --engine both --skip-merge

# Skip loudness normalization
python run.py --engine both --no-normalize

# Skip dynamic compression
python run.py --engine both --no-compress

All CLI options

--input           Path to source markdown (default: ai_landscape_lecture.md)
--output          Output directory (default: ./output)
--engine          kokoro | chatterbox | both
--voice           Kokoro voice id (e.g. am_michael, af_sarah, am_adam, af_nicole)
--speed           Speech rate multiplier (1.0 = normal)
--chapter         Render only the Nth chapter (0-indexed)
--target-words    Target words per Kokoro chapter chunk (default: 1000)
--max-words       Hard cap on words per Kokoro chapter chunk (default: 1200)
--cb-chunk-words  Sub-chunk size for Chatterbox-Turbo (default: 150)
--exaggeration    Chatterbox exaggeration 0.0-1.0 (default: 0.30)
--cfg-weight      Chatterbox CFG weight (default: 0.40)
--temperature     Chatterbox sampling temperature (default: 0.7)
--audio-prompt    Optional reference audio for Chatterbox voice cloning
--no-compress     Skip dynamic-range compression
--no-normalize    Skip loudness normalization
--skip-merge      Render per-chapter WAVs but do not build the final MP3
--skip-samples    (Kokoro) Skip per-voice A/B sample generation

Pipeline stages

Stage 1 — Markdown preprocessing (`src/preprocess.py`)

Strips blockquote metadata and horizontal rules
Replaces fenced code blocks with The document contains a code example demonstrating the concept.
Replaces inline code with a short spoken form
Replaces URLs with Reference link provided in the source material.
Converts tables to a brief spoken summary
Replaces headings with Chapter: <title>.
Replaces bullet/numbered list items with full-stop-separated sentences
Pronounces Greek letters and math symbols

Stage 2 — Intelligent chunking (`src/chunker.py`)

Splits at Chapter: X. markers
For chapters longer than the cap, splits at sentence boundaries while preserving the chapter title in sub-chunk metadata
Configurable target (default 1000) and max (default 1200) word counts

Stage 3 — Kokoro pipeline (`src/kokoro_runner.py`)

Lazy-loads the ONNX model + voice embeddings
Auto-downloads artifacts on first run
Generates per-voice A/B samples for am_michael, af_sarah, am_adam, af_nicole under output/kokoro/voice_samples/
Selects the best voice from the spec's preference list (defaults to am_michael)
Renders one WAV per chapter into output/kokoro/chapters/

Stage 4 — Chatterbox-Turbo pipeline (`src/chatterbox_runner.py`)

Lazy-loads ChatterboxTurboTTS (auto-downloads on first run)
Targets senior-lecturer delivery: low exaggeration, low CFG weight
Sentence-aware sub-chunking at ~150 words to work around the model's long-form truncation behavior
Concatenates sub-segments with 350 ms inter-segment silences
Renders one WAV per chapter into output/chatterbox/chapters/

Stage 5 — Audio post-processing (`src/merger.py`)

Trims leading/trailing silence below 0.5% peak
Concatenates chapters with 600 ms gaps and short crossfades
Applies a feed-forward compressor (3:1 ratio, -6 dB threshold)
Normalizes integrated loudness to -16 LUFS (podcast standard)
Exports the merged audio as 192 kbps MP3 via ffmpeg

Stage 6 — Quality analysis (`src/report.py`)

Aggregates per-chapter and per-engine stats
Computes realtime factor, speaking rate, file size
Builds a side-by-side comparison_report.md with:
- Executive summary
- Pipeline input spec
- Headline metrics table
- Per-voice Kokoro A/B recommendation
- Qualitative observations for each engine
- Per-chapter detail tables
- Use-case recommendation matrix

Performance

On a CPU-only machine (the spec's RTX 4060 accelerates the same code by roughly 5x for Chatterbox; Kokoro is CPU-bound either way):

Engine	Realtime factor (CPU)	Audio per minute of input
Kokoro-82M (ONNX)	~0.9–1.1x	~1 minute
Chatterbox-Turbo	~0.1–0.2x	~6–10 minutes

Both pipelines cache per-chapter WAVs; re-runs only render missing or invalidated chapters.

The full ~45-minute lecture takes approximately:

Kokoro: ~45 minutes of generation
Chatterbox-Turbo: ~6–8 hours of generation (CPU)
GPU (RTX 4060): Chatterbox drops to ~1–2 hours

Adapting to other documents

The pipeline is content-agnostic. To convert a different markdown file:

python run.py --engine both --input path/to/other_document.md

The chunker respects ## headings as chapter boundaries. For documents without ## headings, the entire document becomes a single chapter.

Error handling

The CLI logs all stages and reports failures with the offending chapter and stack trace. Common failure modes:

Error	Cause	Fix
`Failed to download Kokoro artifacts`	Network issue during first run	Manually place `kokoro-v0_19.onnx` and `voices.json` in `~/.cache/ai_audiobook/kokoro/`
`HTTPError 401 from Hugging Face`	Gated model or rate limit	Wait and retry, or pre-download via `huggingface-cli login`
`Couldn't find ffmpeg`	Missing system ffmpeg	`static-ffmpeg` ships a binary under its install dir; it's added to `PATH` automatically by `utils.py`
`UnicodeDecodeError` on voices	Wrong voices file (binary instead of JSON)	Ensure `voices.json`, not `voices.bin`
`Possible clipped samples` warning	Loud peaks after normalization	Benign; downstream MP3 encoder handles clipping

License

This pipeline is provided as-is for research and personal use. Kokoro-82M is released under Apache 2.0; Chatterbox-Turbo is released under MIT by Resemble AI.

Acknowledgements

Kokoro-82M by Hexgrad
kokoro-onnx by thewh1teagle
Chatterbox by Resemble AI

Name		Name	Last commit message	Last commit date
Latest commit History 4 Commits
src		src
.gitignore		.gitignore
README.md		README.md
ai_landscape_lecture.md		ai_landscape_lecture.md
ai_landscape_lecture.mp3		ai_landscape_lecture.mp3
requirements.txt		requirements.txt
run.py		run.py

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

AI Audiobook Generator

Highlights

Repository layout

Setup

1. Requirements

2. Create a virtual environment (optional but recommended)

3. Install dependencies

Usage

Run both engines end-to-end (recommended)

Run a single engine

Choose a Kokoro voice

Adjust speech rate

Render a single chapter (smoke test / debugging)

Tune Chatterbox-Turbo expressiveness

Voice cloning with Chatterbox-Turbo

Skip parts of the pipeline

All CLI options

Pipeline stages

Stage 1 — Markdown preprocessing (`src/preprocess.py`)

Stage 2 — Intelligent chunking (`src/chunker.py`)

Stage 3 — Kokoro pipeline (`src/kokoro_runner.py`)

Stage 4 — Chatterbox-Turbo pipeline (`src/chatterbox_runner.py`)

Stage 5 — Audio post-processing (`src/merger.py`)

Stage 6 — Quality analysis (`src/report.py`)

Performance

Adapting to other documents

Error handling

License

Acknowledgements

About

Releases

Packages

Contributors

Languages

Folders and files

Latest commit

History

Repository files navigation

AI Audiobook Generator

Highlights

Repository layout

Setup

1. Requirements

2. Create a virtual environment (optional but recommended)

3. Install dependencies

Usage

Run both engines end-to-end (recommended)

Run a single engine

Choose a Kokoro voice

Adjust speech rate

Render a single chapter (smoke test / debugging)

Tune Chatterbox-Turbo expressiveness

Voice cloning with Chatterbox-Turbo

Skip parts of the pipeline

All CLI options

Pipeline stages

Stage 1 — Markdown preprocessing (src/preprocess.py)

Stage 2 — Intelligent chunking (src/chunker.py)

Stage 3 — Kokoro pipeline (src/kokoro_runner.py)

Stage 4 — Chatterbox-Turbo pipeline (src/chatterbox_runner.py)

Stage 5 — Audio post-processing (src/merger.py)

Stage 6 — Quality analysis (src/report.py)

Performance

Adapting to other documents

Error handling

License

Acknowledgements

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

Stage 1 — Markdown preprocessing (`src/preprocess.py`)

Stage 2 — Intelligent chunking (`src/chunker.py`)

Stage 3 — Kokoro pipeline (`src/kokoro_runner.py`)

Stage 4 — Chatterbox-Turbo pipeline (`src/chatterbox_runner.py`)

Stage 5 — Audio post-processing (`src/merger.py`)

Stage 6 — Quality analysis (`src/report.py`)