Skip to content

Self-hosting: improved worker and bucket bootstrap #2209

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jun 29, 2025
Merged

Self-hosting: improved worker and bucket bootstrap #2209

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
474bb50
bump image and chart
nicktrn Jun 27, 2025
e1c577d
add webapp sidecar for token bootstrap
nicktrn Jun 28, 2025
62a82e2
bump chart version
nicktrn Jun 28, 2025
5bca4ad
create packets bucket by default
nicktrn Jun 29, 2025
b9e3cc1
fix v4 releaseConcurrency docs for wait.for
nicktrn Jun 29, 2025
4cd1c4e
bump chart
nicktrn Jun 29, 2025
da40f5f
automatically create packets bucket for docker setup
nicktrn Jun 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/self-hosting/docker.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,10 @@ You should change these before deploying to production, especially the password.

### Setup

<Note>
The `packets` bucket is created by default. In case this doesn't work, you can create it manually.
</Note>

1. Login to the dashboard: `http://localhost:9001`

2. Create a bucket named `packets`.
Expand Down
2 changes: 1 addition & 1 deletion docs/upgrade-to-v4.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -761,7 +761,7 @@ You can also now control whether concurrency is released when performing a wait:

```ts
// This will prevent the run from being released back into the queue when the wait starts
await wait.for({ seconds: 10 }, { releaseConcurrency: false });
await wait.for({ seconds: 10, releaseConcurrency: false });
```

The new default behavior allows you to ensure that you can control the number of executing & waiting runs on a queue, and guarantee runs will resume once they are meant to be resumed.
Expand Down
7 changes: 4 additions & 3 deletions hosting/docker/webapp/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@ services:
start_period: 10s

minio:
image: minio/minio:${MINIO_IMAGE_TAG:-latest}
image: bitnami/minio:${MINIO_IMAGE_TAG:-latest}
restart: ${RESTART_POLICY:-unless-stopped}
logging: *logging-config
ports:
Expand All @@ -190,11 +190,12 @@ services:
networks:
- webapp
volumes:
- minio:/data
- minio:/bitnami/minio/data
environment:
MINIO_ROOT_USER: ${MINIO_ROOT_USER:-admin}
MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD:-very-safe-password}
command: server --console-address ":9001" /data
MINIO_DEFAULT_BUCKETS: packets
MINIO_BROWSER: "on"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 5s
Expand Down
4 changes: 2 additions & 2 deletions hosting/k8s/helm/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ apiVersion: v2
name: trigger
description: The official Trigger.dev Helm chart
type: application
version: 4.0.0-beta.7
appVersion: trigger-helm-rc.0
version: 4.0.0-beta.10
appVersion: trigger-helm-rc.1
home: https://trigger.dev
sources:
- https://github.com/triggerdotdev/trigger.dev
Expand Down
13 changes: 11 additions & 2 deletions hosting/k8s/helm/templates/supervisor.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ spec:
{{- with .Values.supervisor.podSecurityContext }}
{{- toYaml . | nindent 8 }}
{{- end }}
{{- if not .Values.webapp.bootstrap.enabled }}
initContainers:
- name: init-shared
image: busybox:1.35
Expand All @@ -83,6 +84,7 @@ spec:
volumeMounts:
- name: shared
mountPath: /home/node/shared
{{- end }}
containers:
- name: supervisor
image: {{ include "trigger-v4.supervisor.image" . }}
Expand Down Expand Up @@ -134,8 +136,11 @@ spec:
- name: TRIGGER_API_URL
value: "http://{{ include "trigger-v4.fullname" . }}-webapp:{{ .Values.webapp.service.port }}"
- name: TRIGGER_WORKER_TOKEN
{{- if .Values.supervisor.bootstrap.enabled }}
value: "file://{{ .Values.supervisor.bootstrap.workerTokenPath }}"
{{- if .Values.webapp.bootstrap.enabled }}
valueFrom:
secretKeyRef:
name: {{ include "trigger-v4.fullname" . }}-worker-token
key: token
{{- else if .Values.supervisor.bootstrap.workerToken.secret.name }}
valueFrom:
secretKeyRef:
Expand Down Expand Up @@ -234,13 +239,16 @@ spec:
{{- with .Values.supervisor.extraEnvVars }}
{{- toYaml . | nindent 12 }}
{{- end }}
{{- if not .Values.webapp.bootstrap.enabled }}
volumeMounts:
- name: shared
mountPath: /home/node/shared
{{- end }}
{{- with .Values.supervisor.securityContext }}
securityContext:
{{- toYaml . | nindent 12 }}
{{- end }}
{{- if not .Values.webapp.bootstrap.enabled }}
volumes:
- name: shared
{{- if .Values.persistence.shared.enabled }}
Expand All @@ -249,6 +257,7 @@ spec:
{{- else }}
emptyDir: {}
{{- end }}
{{- end }}
{{- with .Values.supervisor.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
Expand Down
86 changes: 86 additions & 0 deletions hosting/k8s/helm/templates/webapp.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,39 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ include "trigger-v4.fullname" . }}-webapp
labels:
{{- $component := "webapp" }}
{{- include "trigger-v4.componentLabels" (dict "Chart" .Chart "Release" .Release "Values" .Values "component" $component) | nindent 4 }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: {{ include "trigger-v4.fullname" . }}-webapp-token-syncer
labels:
{{- $component := "webapp" }}
{{- include "trigger-v4.componentLabels" (dict "Chart" .Chart "Release" .Release "Values" .Values "component" $component) | nindent 4 }}
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["create", "get", "update", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: {{ include "trigger-v4.fullname" . }}-webapp-token-syncer
labels:
{{- $component := "webapp" }}
{{- include "trigger-v4.componentLabels" (dict "Chart" .Chart "Release" .Release "Values" .Values "component" $component) | nindent 4 }}
subjects:
- kind: ServiceAccount
name: {{ include "trigger-v4.fullname" . }}-webapp
namespace: {{ .Release.Namespace }}
roleRef:
kind: Role
name: {{ include "trigger-v4.fullname" . }}-webapp-token-syncer
apiGroup: rbac.authorization.k8s.io
---
apiVersion: apps/v1
kind: Deployment
metadata:
Expand All @@ -19,6 +55,7 @@ spec:
labels:
{{- include "trigger-v4.componentSelectorLabels" (dict "Chart" .Chart "Release" .Release "Values" .Values "component" $component) | nindent 8 }}
spec:
serviceAccountName: {{ include "trigger-v4.fullname" . }}-webapp
{{- with .Values.global.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
Expand All @@ -38,6 +75,55 @@ spec:
- name: shared
mountPath: /home/node/shared
containers:
- name: token-syncer
image: bitnami/kubectl:1.28
securityContext:
runAsUser: 1000
runAsNonRoot: true
command:
- /bin/bash
- -c
- |
TOKEN_FILE="/home/node/shared/worker_token"
SECRET_NAME="{{ include "trigger-v4.fullname" . }}-worker-token"
NAMESPACE="{{ .Release.Namespace }}"

echo "Token syncer starting..."
echo "Monitoring: $TOKEN_FILE"
echo "Target secret: $SECRET_NAME"

while true; do
if [ -f "$TOKEN_FILE" ]; then
TOKEN=$(cat "$TOKEN_FILE")
if [ ! -z "$TOKEN" ]; then
echo "Token file found, creating/updating secret..."

# Create or update the secret
kubectl create secret generic "$SECRET_NAME" \
--from-literal=token="$TOKEN" \
--namespace="$NAMESPACE" \
--dry-run=client -o yaml | kubectl apply -f -

if [ $? -eq 0 ]; then
echo "Secret successfully created/updated"
# Continue monitoring for updates
sleep 30
else
echo "Failed to create/update secret, retrying in 5s..."
sleep 5
fi
else
echo "Token file exists but is empty, waiting..."
sleep 2
fi
else
echo "Waiting for token file..."
sleep 2
fi
done
volumeMounts:
- name: shared
mountPath: /home/node/shared
Comment on lines +78 to +126
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Hard-code of worker-token path risks drift – parameterise it

The side-car script uses:

TOKEN_FILE="/home/node/shared/worker_token"

but the actual file location is configurable through
.Values.webapp.bootstrap.workerTokenPath, exported to the main container
as TRIGGER_BOOTSTRAP_WORKER_TOKEN_PATH.

Keep the two in sync to avoid silent failures when the value is overridden:

-              TOKEN_FILE="/home/node/shared/worker_token"
+              TOKEN_FILE="{{ .Values.webapp.bootstrap.workerTokenPath }}"

Optionally pass the same value via an env-var so the shell stays readable.

🧰 Tools
🪛 YAMLlint (1.37.1)

[error] 90-90: trailing spaces

(trailing-spaces)

[error] 94-94: trailing spaces

(trailing-spaces)

[error] 100-100: trailing spaces

(trailing-spaces)

[error] 106-106: trailing spaces

(trailing-spaces)

🤖 Prompt for AI Agents 
In hosting/k8s/helm/templates/webapp.yaml between lines 78 and 126, the
token-syncer container script hardcodes the token file path as
"/home/node/shared/worker_token", which can cause mismatches if the actual path
is overridden by the .Values.webapp.bootstrap.workerTokenPath Helm value. To fix
this, parameterize the token file path by passing the configured path into the
sidecar container via an environment variable, ideally reusing the
TRIGGER_BOOTSTRAP_WORKER_TOKEN_PATH value, and update the script to reference
this environment variable instead of the hardcoded path to ensure consistency
and prevent silent failures.

- name: webapp
securityContext:
{{- toYaml .Values.webapp.securityContext | nindent 12 }}
Expand Down
3 changes: 3 additions & 0 deletions hosting/k8s/helm/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -476,6 +476,9 @@ s3:
rootUser: "admin"
rootPassword: "very-safe-password"

# The required "packets" bucket is created by default.
defaultBuckets: "packets"

persistence:
enabled: true
size: 10Gi
Expand Down