From a628ba161ed46625985432a7f97d2655a88dc42b Mon Sep 17 00:00:00 2001
From: basil-k-aji-dev <70605804+basil-k-aji-dev@users.noreply.github.com>
Date: Sun, 13 Sep 2026 23:51:57 +0530
Subject: [PATCH] fix: install cuDNN 8 compat libraries in the CUDA Docker
 image
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The CUDA base image is pytorch/pytorch:2.8.0-cuda12.8-cudnn9-runtime, so it
ships cuDNN 9. CTranslate2 — WhisperX and faster-whisper — links cuDNN 8, and
its absence aborts the backend process outright rather than raising (#1371).

scripts/setup.py side-loads the cuDNN 8 libraries for source installs, but the
Dockerfile never did, so every CTranslate2 ASR engine was unavailable in Docker
and the demo synthesis timed out with libcudnn_ops_infer.so.8 missing.

Install the same nvidia-cudnn-cu12==8.9.7.29 shim during the image build,
deriving the target from sys.prefix so it matches where backend/core/cudnn8.py
searches rather than hardcoding the conda path — sys.prefix differs between the
conda-based CUDA image and the ROCm venv. Guarded to GPU_FLAVOR=cuda, since
ROCm does not use cuDNN, and --no-deps keeps the base image's torch stack
untouched. A post-install assert fails the build if no .so.8 libraries landed,
rather than letting it resurface as the same runtime warning.

Fixes #2050
---
 deploy/Dockerfile | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/deploy/Dockerfile b/deploy/Dockerfile
index 0666abcc9..49f1360e4 100644
--- a/deploy/Dockerfile
+++ b/deploy/Dockerfile
@@ -103,6 +103,27 @@ RUN python3 -c "import os, torch, torchaudio, torchvision; \
     import torchvision.ops; torchvision.ops.nms; \
     print('torchvision C++ ops resolve against this torch')"
 
+# CTranslate2 (WhisperX, faster-whisper) links cuDNN 8, but the CUDA base image
+# ships cuDNN 9, so libcudnn_ops_infer.so.8 is absent and loading it aborts the
+# backend process outright rather than raising (#1371). scripts/setup.py
+# side-loads the cuDNN 8 libraries for source installs; the image needs the same
+# shim or Docker users lose every CTranslate2 ASR engine (#2050).
+#
+# The target is derived from sys.prefix rather than hardcoded: backend/core/
+# cudnn8.py looks for <sys.prefix>/lib/pythonX.Y/site-packages/cudnn8_compat,
+# and sys.prefix differs between the conda-based CUDA image and the ROCm venv.
+# --no-deps keeps this to the cuDNN wheels alone, leaving the base image's torch
+# stack untouched. Skipped for ROCm, which does not use cuDNN.
+RUN if [ "$GPU_FLAVOR" = "cuda" ]; then \
+      target="$(python3 -c "import os, sys; print(os.path.join(sys.prefix, 'lib', 'python%d.%d' % sys.version_info[:2], 'site-packages', 'cudnn8_compat'))")" && \
+      uv pip install --python "$(command -v python3)" --no-cache --no-deps \
+        --target "$target" nvidia-cudnn-cu12==8.9.7.29 && \
+      python3 -c "import os, sys; d = os.path.join(sys.prefix, 'lib', 'python%d.%d' % sys.version_info[:2], 'site-packages', 'cudnn8_compat', 'nvidia', 'cudnn', 'lib'); \
+        libs = [f for f in os.listdir(d) if '.so.8' in f]; \
+        assert libs, 'cudnn8_compat installed but no .so.8 libraries in ' + d; \
+        print('cuDNN 8 compat libraries: %d' % len(libs))"; \
+    fi
+
 # Copy application source
 COPY backend/ ./backend/
 COPY omnivoice/ ./omnivoice/