fix(docker): install cuDNN 8 compat libraries in the CUDA Docker image - #2072
basil-k-aji-dev wants to merge 1 commit into
Conversation
The CUDA base image is pytorch/pytorch:2.8.0-cuda12.8-cudnn9-runtime, so it ships cuDNN 9. CTranslate2 — WhisperX and faster-whisper — links cuDNN 8, and its absence aborts the backend process outright rather than raising (debpalash#1371). scripts/setup.py side-loads the cuDNN 8 libraries for source installs, but the Dockerfile never did, so every CTranslate2 ASR engine was unavailable in Docker and the demo synthesis timed out with libcudnn_ops_infer.so.8 missing. Install the same nvidia-cudnn-cu12==8.9.7.29 shim during the image build, deriving the target from sys.prefix so it matches where backend/core/cudnn8.py searches rather than hardcoding the conda path — sys.prefix differs between the conda-based CUDA image and the ROCm venv. Guarded to GPU_FLAVOR=cuda, since ROCm does not use cuDNN, and --no-deps keeps the base image's torch stack untouched. A post-install assert fails the build if no .so.8 libraries landed, rather than letting it resurface as the same runtime warning. Fixes debpalash#2050
 The PR appears safe to merge; no blocking defect was identified.
|
ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review. 📝 WalkthroughWalkthroughThe Docker runtime stage now installs pinned cuDNN 8 compatibility libraries for CUDA images. It derives the target site-packages path and verifies the installed ChangesCUDA cuDNN compatibility
Priority: ➖ Normal Estimated code review effort: 2 (Simple) | ~10 minutes Change: Bug fix · Severity of issue fixed: Medium Suggested reviewers: Merge Risk: ⚪ Minimal · up to The CUDA image installs and verifies the cuDNN 8 compatibility libraries required by the affected ASR engines, with no merge-blocking issue identified. 🚥 Pre-merge checks | ✅ 8 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (8 passed)
Full details: Local-First GuaranteeExplanation The PR adds a required outbound call to PyPI during CUDA image builds. In Resolution Remove the required PyPI dependency from the Docker build. Vendor the pinned cuDNN 8 wheel in an approved local build input, or copy it from an approved prebuilt image/artifact and install it with a local-only
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
basil-k-aji-dev
changed the title
|
Thanks — title updated to On the local-first warning, I think it is being applied to the wrong layer, but I would rather lay out the reasoning than quietly dismiss it. The gate in CONTRIBUTING reads:
That sits among runtime properties — logging, persisted secrets, home paths — so I read it as constraining what the running application does on a user's machine, not what the image build fetches. This change adds nothing at runtime; it only puts files into the image. At build time the Dockerfile already reaches the network in four places:
My line 119 is the same mechanism as 64 and 90, pinned to the exact version Vendoring the wheel as the bot suggests would mean carrying ~700 MB of platform-specific CUDA binaries in-tree, which seems a worse trade than one pinned install alongside the four that already run. Happy to be overruled — if you would prefer it vendored, sourced from a prebuilt artifact, or made opt-in behind a build arg, say which and I will rework it. |
1 participant
Fixes #2050
The CUDA base image is
pytorch/pytorch:2.8.0-cuda12.8-cudnn9-runtime, so it ships cuDNN 9. CTranslate2 — WhisperX and faster-whisper — links cuDNN 8, and per the note inbackend/core/cudnn8.pyits absence aborts the backend process outright rather than raising (#1371).scripts/setup.pyside-loads the cuDNN 8 libraries for source installs, but the Dockerfile never did. Docker users therefore lose every CTranslate2 ASR engine, which surfaces as the demo synthesis timing out and:Change
Install the same
nvidia-cudnn-cu12==8.9.7.29shim during the image build.Two details worth flagging:
The target is derived from
sys.prefix, not hardcoded.backend/core/cudnn8.py::compat_dirs()searches<sys.prefix>/lib/pythonX.Y/site-packages/cudnn8_compat/nvidia/cudnn/lib, andsys.prefixdiffers between the conda-based CUDA image and the ROCm venv. Deriving it keeps the install and the lookup in agreement if the base image's Python version or layout moves. I verified the computed target is exactly the parent of the directorycompat_dirs()looks in.A post-install assert fails the build if no
.so.8libraries landed. Without it, a silently empty install would reappear as this same runtime warning, which is what made the original report hard to place.Guarded to
GPU_FLAVOR=cuda— ROCm does not use cuDNN — and--no-depskeeps the install to the cuDNN wheels alone, leaving the base image's torch stack untouched, consistent with the existing torch-clobber guard directly above.Quality gates
uv pip installsteps in this Dockerfile. No new runtime outbound calls.Testing
docker build --check -f deploy/Dockerfile .passes — the only warning isInvalidBaseImagePlatform, because I am on arm64 and the base image is amd64.What I could not run: I could not build the image or run
uv run pytest backend/ -x -qon this machine — it is an arm64 Raspberry Pi, and the CUDA base image is amd64-only. So the install step itself is unexercised here and I would rely on CI for it. What I did verify directly is the path logic, since that is the part most likely to be wrong:The pinned version matches the one
scripts/setup.pyalready installs, so it is not a new dependency choice.The reporter in #2050 confirmed a hardcoded variant of this fix works in a derived image; this PR generalises it and adds the verification step.
The CUDA Docker image now installs
nvidia-cudnn-cu12==8.9.7.29into thecudnn8_compatpath required by CTranslate2-based WhisperX and faster-whisper engines. The install runs only forGPU_FLAVOR=cuda, uses--no-deps, and verifies that.so.8libraries are present. Full image builds and backend tests were not run because the available machine is arm64 while the CUDA image is amd64.