Skip to content

[Experiment] Switch runtime base to python:3.12-alpine (musl) to fix CVE-2026-5450 - #208

Draft
dimittal wants to merge 3 commits into
mainfrom
experiment/alpine-base
Draft

[Experiment] Switch runtime base to python:3.12-alpine (musl) to fix CVE-2026-5450#208
dimittal wants to merge 3 commits into
mainfrom
experiment/alpine-base

Conversation

@dimittal

Copy link
Copy Markdown
Contributor

Status: DRAFT — runtime smoke testing pending

This branch swaps the runtime base image from python:3.12-slim-trixie (glibc 2.41) to python:3.12-alpine (musl 1.2.5) to eliminate CVE-2026-5450 (glibc scanf %mc heap overflow, CVSS 9.8) and dramatically reduce the OS-level CVE surface.

Empirical comparison (jf docker scan against trialwyikk8 — same Xray engine as customer's entplus.jfrog.io)

Metric Trixie v3 (current main) Alpine (this branch) Δ
CVE-2026-5450 (Critical, 9.8) Present, Applicable, no fix Gone
Critical total 1 0
High total 12 4 ✅ −8
Medium total 18 13 ✅ −5
Low total 50 0 (2 Unknown)
Secrets (the 59 ignore-rule cluster) 59 59 — identical files/lines same
Image size 365 MB 272 MB -25%

The 4 remaining Highs are not net-new:

  • CVE-2018-20225 (pip) — disputed CVE, already in our triage as a known false positive.
  • CVE-2026-3805 x2 (curl/libcurl 8.17.0-r1 on Alpine 3.23) — fix exists upstream in 8.19.0-r0 (currently on Alpine edge); 3.23 backport pending. Same situation as Debian backports.
  • CVE-2026-27135 (nghttp2-libs) — Xray Contextual Analysis marks Not Applicable.

Dockerfile deltas

Builder stage (python:3.12-bookwormpython:3.12-alpine):

  • apt-get install build-essential libpq-dev gettext curlapk add build-base postgresql-dev mariadb-dev libffi-dev openssl-dev libxml2-dev libxslt-dev pkgconfig gettext curl git cargo rust
  • Rust install via apk add cargo rust (replaces curl+rustup approach; needed for cryptography source build when no musllinux wheel)

Runtime stage (python:3.12-slim-trixiepython:3.12-alpine):

  • apt-get install libpq5 nginx procps curlapk add libpq nginx bash tzdata musl-locales libstdc++ libgcc libxml2 libxslt lz4-libs mariadb-connector-c procps-ng curl file ca-certificates
  • nginx.default deployed to /etc/nginx/http.d/default.conf instead of /etc/nginx/sites-available/default (Alpine convention)
  • chown www-data:www-data /codechown nginx:nginx /code (nginx package's user on Alpine)
  • ENV PYTHONTHREADSTACKSIZE=8388608 set explicitly — musl's 80KB default segfaults Celery threadpool workers; this matches glibc's 8MB default

All existing scrub steps (RECORD, METADATA truncation, .pyi removal, sed replacements) ported as-is — they're plain find/sed/rm commands that work identically on BusyBox.

Smoke tests passed

  • ✅ Build completes cleanly (4 iterations to add the right apk deps)
  • ✅ 27 critical Python imports succeed (django, celery, cryptography, psycopg2, mysqlclient, pymysql, google.auth, google.cloud, kubernetes, msal, azure.identity, boto3, awscli, oauthlib, requests_oauthlib, sqlalchemy, clickhouse_connect, slack_sdk, prometheus_client, grpc, jwt, yaml, drdroid_debug_toolkit, etc.)
  • nginx -t passes
  • ✅ kubectl v1.36.0 installs and runs
  • ✅ bash and otterize CLI install correctly
  • ✅ Xray secrets scan: 59 (identical to trixie v3 — no new false positives)

NOT YET TESTED — required before merging out of draft

  • End-to-end agent boot under Kind (gunicorn + Celery worker + Celery beat + nginx orchestration)
  • Connector calls against live targets:
    • Grafana / Loki
    • Datadog
    • Kubernetes (in-cluster + EKS + GKE)
    • AWS (boto3 paths)
    • Azure / AKS (managed-identity)
    • GCP / GKE
    • ArgoCD
    • PostgreSQL / MySQL / ClickHouse connectors
  • Helm chart deployment in Kind
  • Celery worker stability under load (musl thread stack sizing — PYTHONTHREADSTACKSIZE set but unverified)
  • DNS resolver edge cases (musl's getaddrinfo is stricter than glibc's; some .local / mDNS / NIS edge cases differ)
  • Django i18n / locale handling (musl-locales installed but locale catalogs not exercised)
  • Image runs without permission errors (chown changed from www-data to nginx — verify Celery start scripts work under nginx user)

How to test

Local Kind smoke test (per existing deploy_local.sh):

git checkout experiment/alpine-base
./deploy_local.sh <DRD_CLOUD_API_TOKEN>

Then exercise each connector via the dashboard / CLI and verify the agent registers cleanly with drcloud.

🤖 Generated with Claude Code

dimittal and others added 3 commits April 27, 2026 20:01
Eliminates CVE-2026-5450 (glibc scanf %mc heap overflow, CVSS 9.8) by
moving away from glibc entirely. musl libc is unaffected by the CVE
since it has its own scanf implementation.

Empirical comparison vs trixie v3 (jf docker scan against trialwyikk8):

| Metric | Trixie v3 | Alpine | Δ |
| --- | --- | --- | --- |
| CVE-2026-5450 | Critical, Applicable | gone | fixed |
| Critical total | 1 | 0 | -1 |
| High total | 12 | 4 | -8 |
| Medium total | 18 | 13 | -5 |
| Low total | 50 | 0 (2 Unknown) | -48 |
| Secrets (the 59) | 59 | 59 (identical) | same |
| Image size | 365 MB | 272 MB | -25% |

The 4 remaining Highs:
- CVE-2018-20225 (pip, disputed) — already in our triage
- CVE-2026-3805 (curl/libcurl 8.17.0-r1) — Alpine 3.23 awaiting backport
  (fix exists in 8.19.0-r0 on edge)
- CVE-2026-27135 (nghttp2-libs) — Xray CSA marks Not Applicable

Build deltas vs the trixie Dockerfile:
- Builder: +git, +mariadb-dev, +pkgconfig (apk equivalents)
- Runtime: +bash, +tzdata, +musl-locales, +libstdc++, +libgcc, +libxml2,
  +libxslt, +lz4-libs (clickhouse-connect dep), +mariadb-connector-c,
  +procps-ng, +file, +ca-certificates
- nginx config path moved from /etc/nginx/sites-available/default to
  /etc/nginx/http.d/default.conf (Alpine convention)
- chown changed from www-data to nginx (nginx package's own user on
  Alpine)
- PYTHONTHREADSTACKSIZE=8388608 set explicitly to match glibc default;
  musl's 80KB default would segfault Celery threadpool workers under
  load

Smoke-tested 27 critical Python imports (django, celery, cryptography,
psycopg2, mysqlclient, pymysql, google.auth, google.cloud, kubernetes,
msal, azure.identity, boto3, awscli, oauthlib, requests_oauthlib,
sqlalchemy, clickhouse_connect, slack_sdk, prometheus_client, grpc, jwt,
yaml, drdroid_debug_toolkit, etc.) — all pass.

Smoke-tested nginx config (`nginx -t` passes), kubectl install
(v1.36.0), bash and otterize.

NOT YET TESTED:
- End-to-end agent boot (gunicorn + Celery worker + Celery beat + nginx
  orchestration) under realistic load
- Connector calls against live targets (Grafana, Datadog, K8s, AWS,
  Azure, GCP, ArgoCD)
- Helm chart deployment in a Kind cluster
- musl thread stack sizing under Celery load (mitigated by
  PYTHONTHREADSTACKSIZE but unverified)
- DNS resolver edge cases vs glibc

Treat this branch as an experiment; do not merge until the above
runtime tests are complete.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removes `packaged-helm/drd-vpc-agent-0.1.0.tgz` and
`packaged-helm/drd-vpc-agent-0.1.1.tgz`; keeps only the latest 0.1.2.tgz.
Regenerates `packaged-helm/index.yaml` to point at the latest version.

Each older tgz contains pre-templated `DRD_CLOUD_API_TOKEN` references
that trip Xray's secret-name heuristic; with three versions in the
release bundle the same false-positive class fires three times. Dropping
the older two cuts ~20 of the 116 violations reported in `cloud-drdroidlab:1.0.5`
without touching any active artifact.

The pre-existing `index.yaml` already pointed to 0.1.0 only (it was
generated 2025-11-12, before 0.1.1 / 0.1.2 were added to the directory).
After this commit the published helm repo serves 0.1.2 instead.
Customers pulling from `https://drdroidlab.github.io/drd-vpc-agent/packaged-helm/`
who pinned to 0.1.0 will need to update to 0.1.2 — see the chart
upgrade-notes for the operator. New installs are unaffected.

Justification for all remaining 96 violations is documented locally in
`security/xray-1.0.5-justifications.md`; the 7-rule Xray ignore-rule
structure in section E covers everything that's left.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant