Repository navigation
Expand file tree
/
Copy pathDockerfile.agent
More file actions
214 lines (202 loc) · 11.1 KB
/
Copy pathDockerfile.agent
File metadata and controls
214 lines (202 loc) · 11.1 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
# hoophq/hoopagent — the hoop agent image, in two slim flavours, each also
# published with the alcatraz NER model baked in.
#
# One Dockerfile, four final stages selected with `docker build --target`. They
# share the `unpack` stage, so release-tarball handling exists in exactly one
# place, and the `models` stage, so one model download serves both -alcatraz
# targets:
#
# --target minimal -> hoophq/hoopagent:<version>-minimal
# --target distroless -> hoophq/hoopagent:<version>-distroless
# --target minimal-alcatraz -> hoophq/hoopagent:<version>-minimal-alcatraz
# --target distroless-alcatraz -> hoophq/hoopagent:<version>-distroless-alcatraz
#
# There is deliberately no unsuffixed tag and no `:latest` — every deployment
# names the flavour it wants explicitly, and CI always passes --target. A bare
# `docker build -f Dockerfile.agent` with no --target builds the stage declared
# LAST, `minimal-alcatraz`: it keeps a shell and is a superset of the other
# three, so an accidental default is never the restrictive one.
#
# ---------------------------------------------------------------------------
# LICENSE BRIGHT LINE — read before adding a stage
#
# hoophq/hoopagent is a clean-only image repository: no layer published under it
# may carry an AGPL- or SSPL-licensed component, so that the repository has no
# such component anywhere in its history. Every flavour here is built from a
# stock Ubuntu or distroless rootfs plus the `hoop` binary, and carries none.
# The model weights the -alcatraz stages add are served by hoop's own mirror and
# carry no such component either.
#
# This is why no flavour is built FROM hoophq/agent-tools. That base bundles
# third-party database clients including an SSPL-licensed MongoDB shell, and
# deriving from it would put those layers into this repository permanently. Agent
# deployments that need the bundled tooling or RDP use hoophq/hoopdev instead —
# with the model mounted from a volume, since no hoopdev flavour bakes it in.
#
# Do NOT add a stage here that derives from agent-tools, or that installs
# database clients, without first checking the license policy in
# scripts/ci/check-forbidden-licenses.py.
#
# ---------------------------------------------------------------------------
# Choosing a flavour
#
# minimal Ubuntu 24.04 LTS rootfs + the `hoop` binary and nothing else.
# distroless Distroless static rootfs + the `hoop` binary and nothing else.
#
# Both serve every connection type the agent handles in-process (tcp, postgres,
# mysql, mssql, mongodb, httpproxy, ssh, ssm) and run as uid 10001. Neither
# supports custom / command-line connections (no shell or client tooling to
# exec), nor RDP (that needs the `hoop_rs` Rust agent, which is not shipped
# here). Use hoophq/hoopdev for those.
#
# distroless carries no OS package manager and no shell, so it reports zero
# OS-package CVEs and is roughly two thirds the size of minimal — at the cost of
# `kubectl exec <pod> -- sh` being impossible. Use minimal when that
# debuggability matters, distroless when the smallest possible surface does.
#
# ---------------------------------------------------------------------------
# The -alcatraz flavours
#
# DLP_PROVIDER=alcatraz serves PERSON, LOCATION and NRP from an ONNX model that
# runs in-process on the agent. The NER backend is Offline: it never fetches at
# runtime, so the model has to be on disk already, and these stages are that
# disk. Each adds ~250MB of weights and one env var to the flavour it derives
# from — binary, user, CMD and invocation contract are untouched, so a
# deployment moves between `-minimal` and `-minimal-alcatraz` by editing the tag.
#
# Nothing here installs or links alcatraz: that dependency lives in libhoop's
# go.mod. The manifest names the files and their layout, so no model id,
# revision, digest or alcatraz version appears in this repository.
#
# Deriving in-graph rather than FROM a published tag is what makes the four
# builds one job: the -alcatraz tags share their base layers with the plain tags
# byte for byte, the model is fetched once for both, and no build waits on a
# registry that has not caught up with the push before it.
#
# ---------------------------------------------------------------------------
# Invocation contract — IDENTICAL across all four flavours, so a deployment can
# switch tags without touching its manifest:
#
# - no ENTRYPOINT; the command is `CMD ["hoop", "start", "agent"]` resolved
# through /app on PATH.
# - override with `command: ["hoop", "start", "agent", ...]`, never with a
# bare `args:` — the latter replaces CMD entirely on every flavour.
#
# ---------------------------------------------------------------------------
# Build (from the repo root, with release tarballs in dist/binaries/):
# GOOS=linux GOARCH=arm64 make build-go build-tar-files
# docker build -f Dockerfile.agent --target minimal -t hoophq/hoopagent:dev-minimal .
# docker build -f Dockerfile.agent --target minimal-alcatraz -t hoophq/hoopagent:dev-minimal-alcatraz .
# Ubuntu 24.04 LTS (noble). Pinned deliberately, same policy as ./Dockerfile,
# so builds are reproducible; bump it in a dedicated PR.
ARG UBUNTU_IMAGE=ubuntu:noble-20260730.1
ARG DISTROLESS_IMAGE=gcr.io/distroless/static-debian12:nonroot
ARG CURL_IMAGE=curlimages/curl:8.11.1
# Hoop's mirror of the model files, served straight from the bucket. The
# objects are public-read (the bucket does not grant ListBucket, so the keys
# have to be known — the manifest below is what names them), which is what lets
# the -alcatraz targets build outside CI. An ARG so an air-gapped build can swap
# in its own.
#
# A moving alias, not a revision: rebuilding an old tag may pick up a newer
# model, and model/binary skew is caught when the agent loads. In exchange
# nothing here can drift out of sync with alcatraz's pin table.
ARG ALCATRAZ_MODEL_ORIGIN=https://hoopartifacts.s3.us-east-1.amazonaws.com/alcatraz/current
# --- Stage: unpack the release tarball ---------------------------------------
# A separate stage so the ~200MB of release tarballs in dist/binaries/ never
# reaches a layer of any published flavour (a `rm -rf` in a later RUN would not
# shrink it — the COPY layer keeps the bytes forever).
#
# Pinned to BUILDPLATFORM and keyed off TARGETARCH rather than `uname -m`:
# these images are built as single multiarch jobs, and only the runtime stages
# should be emulated. `make build-tar-files` publishes both the Go-style
# (amd64/arm64) and uname-style (x86_64/aarch64) archive names, so TARGETARCH
# matches directly.
#
# The tarball also carries `hoop_rs` (the Rust agent that serves RDP); only
# `hoop` is extracted, since no flavour here ships it. Naming the member
# explicitly makes a tarball layout change fail the build loudly instead of
# silently producing an image with no agent in it.
FROM --platform=${BUILDPLATFORM} ${UBUNTU_IMAGE} AS unpack
ARG TARGETARCH
COPY dist/binaries/ /tmp/binaries/
RUN mkdir -p /out && \
tar -xf /tmp/binaries/hoop_*_Linux_${TARGETARCH}.tar.gz -C /out/ ./hoop && \
chown root:root /out/hoop && \
chmod 755 /out/hoop
# --- Stage: fetch the alcatraz model -----------------------------------------
# BUILDPLATFORM: the weights are architecture-independent, so a multiarch build
# fetches them once instead of twice under QEMU, and the registry stores one
# layer. Only the -alcatraz targets depend on this stage; `--target minimal`
# never runs it.
FROM --platform=${BUILDPLATFORM} ${CURL_IMAGE} AS models
ARG ALCATRAZ_MODEL_ORIGIN
# curlimages/curl cannot write under /opt. Throwaway stage only.
USER root
WORKDIR /opt/alcatraz/models
# -f: an origin answering 404 must fail the build, not save the error page as a
# model file. --create-dirs: the manifest names a subdirectory that does not
# exist yet. a+rX: the runtime user is unprivileged and has to traverse the
# directories.
#
# A digest mismatch is not transient. The one benign cause is a build reading
# the manifest mid-publish, between the new checksums.txt and the weights it
# names — retry it, never soften this step.
RUN set -eu; \
curl -fsSL "${ALCATRAZ_MODEL_ORIGIN}/checksums.txt" -o checksums.txt; \
while read -r _ path; do \
[ -n "$path" ] || continue; \
curl -fsSL "${ALCATRAZ_MODEL_ORIGIN}/${path}" --create-dirs -o "$path"; \
done < checksums.txt; \
sha256sum -c checksums.txt; \
chmod -R a+rX /opt/alcatraz/models
# --- Flavour: distroless -----------------------------------------------------
# The static rootfs already ships the CA bundle the agent needs for grpcs://,
# so nothing is installed here. uid 10001 is set numerically: distroless has no
# useradd, and the agent never resolves a passwd entry (config comes from the
# environment, see agent/config.Load).
FROM ${DISTROLESS_IMAGE} AS distroless
COPY --from=unpack /out/hoop /app/hoop
ENV PATH="/app:${PATH}"
USER 10001:10001
# No init shim: the agent is PID 1 and handles SIGINT/SIGTERM itself
# (agent.handleOsInterrupt). It forks no children in this flavour.
CMD ["hoop", "start", "agent"]
# --- Flavour: distroless + alcatraz model ------------------------------------
# COPY lands root-owned and a+rX from the models stage, so uid 10001 can read it
# without a USER dance. The models directory is the parent holding one
# subdirectory per model id, not the model's own directory.
FROM distroless AS distroless-alcatraz
COPY --from=models /opt/alcatraz/models /opt/alcatraz/models
ENV ALCATRAZ_NER_MODEL_PATH=/opt/alcatraz/models
# --- Flavour: minimal (Ubuntu 24.04 LTS) -------------------------------------
FROM ${UBUNTU_IMAGE} AS minimal
ENV DEBIAN_FRONTEND=noninteractive
# ca-certificates is the one runtime dependency the binary cannot carry itself:
# the agent dials the gateway over TLS (grpcs://) and, unless HOOP_TLSCA pins a
# CA, Go verifies the server against the system trust store — which the ubuntu
# base rootfs does not include. Nothing else is installed.
RUN groupadd --gid 10001 hoop && \
useradd --uid 10001 --gid 10001 --shell /sbin/nologin hoop && \
apt-get update -y && \
apt-get install -y --no-install-recommends ca-certificates && \
rm -rf /var/lib/apt/lists/* /var/cache/apt/archives
# Ownership and mode are already root:root 0755 from the unpack stage.
COPY --from=unpack /out/hoop /app/hoop
ENV PATH="/app:${PATH}"
USER hoop
# No init shim (tini is not installed) — see the distroless note above.
CMD ["hoop", "start", "agent"]
# --- Flavour: minimal + alcatraz model ---------------------------------------
# Declared last on purpose: see the note about a bare build at the top.
#
# No verification step here: distroless has no shell to run one in, and the only
# build this could catch is one where COPY --from lost bytes the models stage
# had just checksummed. The check that matters runs in CI against the artifact
# that landed in the registry, per architecture, by exporting the container
# filesystem — see scripts/ci/verify-alcatraz-image.sh. The manifest ships with
# the model, so that same command answers "is this intact" for any copy of the
# image, running or not.
FROM minimal AS minimal-alcatraz
COPY --from=models /opt/alcatraz/models /opt/alcatraz/models
ENV ALCATRAZ_NER_MODEL_PATH=/opt/alcatraz/models