A collection of service endpoints to support Marketing Operations campaign creation and management.
/livez:GET— checks that the service is alive (liveness probe). Returns200with atext/plainbody ofOK. Does not depend on database availability./readyz:GET— checks that the service is able to take inbound requests (readiness probe), including a PostgreSQL connectivity check. Returns200with atext/plainbody ofOKwhen ready, or503when not ready./metrics:GET— Prometheus metrics in the text exposition format. Served on the same port as the API, and available in no-database mode and during a cold start (a scrape target that only appears once the database is up is exactly backwards). See Metrics below.
These three endpoints are unauthenticated and are excluded from the
generated public API documentation. They are also absent from the Helm
chart's HTTPRoute and Heimdall RuleSet, so they are reachable only
in-cluster (kubelet probes and the Prometheus scraper), never through
the public gateway.
GET /metrics serves the Prometheus text exposition format. The
endpoint takes no configuration — there is no environment variable
to enable it and no separate metrics port. It is independent of the
OTEL_* settings below: OTEL_METRICS_EXPORTER defaults to none, so
wiring these instruments to the OTLP pipeline would leave /metrics
empty in the default deployment.
Service metrics:
| Metric | Type | Labels | What it answers |
|---|---|---|---|
campaign_dispatch_total |
counter | platform, outcome |
Are campaigns actually landing on each ad platform? outcome is one of success, skipped, failure, panic — panic is separate because it is a bug in this service, not an upstream refusal. |
campaign_job_transitions_total |
counter | status |
Dispatch jobs reaching each state (running, succeeded, partial, failed). A growing gap between running and the terminal states means jobs are getting stuck — the terminal transition is recorded only when the status write actually persisted, so a job stuck by a failed write leaves the gap open rather than closing it. |
campaign_upstream_calls_total |
counter | platform, operation, outcome |
Upstream ad-platform API call volume and error rate. |
campaign_upstream_call_duration_seconds |
histogram | platform, operation, outcome |
Upstream ad-platform latency. Timed only after the pre-platform guards pass, so local refusals do not drag the quantiles toward zero. Bucketed on the service's own call budgets (10ms…45s, with edges on the 20s/30s/45s ceilings) rather than the OTel default, whose ms-scale boundaries would collapse every healthy call into one bucket. |
campaign_db_pool_* |
gauge / counter | none | Database pool health: acquired, idle, total, max and newly-established connections, plus canceled and empty acquires. Exported only when a pool is actually wired — see below. |
Plus the standard Go runtime and process collectors.
Label cardinality. No metric here carries a campaign id, brief id,
project id, job id, account id or URL as a label value — an unbounded
label creates one retained time series per distinct value and is the
classic way to take a Prometheus server down. platform is mapped
through a closed provider set and anything outside it collapses to
unknown; the outcome labels are closed enums; and operation passes a
shape guard that admits short lower-snake tokens and degrades anything
id-shaped or derived to unknown. No metric name, label
or help string carries a credential, DSN or token.
Absent pool metrics are meaningful. When no database is wired (no-DB
mode, or a cold start before the pool opens) the campaign_db_pool_*
series are not exported at all, rather than exported as zeroes. A
zero is a measurement: reporting max_connections=0 for a service
running without a database is indistinguishable from a pool that has
collapsed, and would fire a false exhaustion alert.
Scraping. The chart sets prometheus.io/scrape,
prometheus.io/path and prometheus.io/port on the pod template by
default, so a discovery-based collector picks the pod up with no further
configuration. prometheus.io/port is derived from service.port in
the template rather than hardcoded in values.yaml, so the scrape port
cannot drift from the port the container listens on. A deployment that
overrides podAnnotations replaces the map, so it must re-declare
prometheus.io/scrape and prometheus.io/path to keep being scraped.
Configuration priority: CLI flags > environment variables > defaults.
When any PostgreSQL setting is supplied, the set must be complete or
the process exits non-zero. Fully omitting all database settings is
allowed for unit tests / metadata-only local runs (no-DB mode; /readyz
stays process-ready without a pool). In-cluster they are typically
injected from the ExternalSecret-managed Kubernetes secret
(lfx-v2-campaign-service-secrets in namespace
lfx-v2-campaign-service; keys host, port, username,
password, dbname, engine, credential-encryption-key).
PGHOST(secret keyhost) — PostgreSQL hostnamePGUSER(secret keyusername) — PostgreSQL usernamePGPASSWORD(secret keypassword) — PostgreSQL password (never logged)PGDATABASE(secret keydbname) — PostgreSQL database nameCREDENTIAL_ENCRYPTION_KEY(secret keycredential-encryption-key) — base64-encoded 32-byte AES-256 key used to encrypt ad-platform connection credentials at rest. Required whenever a database is configured. Rollout prerequisite: provision this key onlfx-v2-campaign-service-secrets(via ExternalSecret sync) before deploying the chart revision that references it; otherwise the pod stays inCreateContainerConfigError.
The service composes the DSN in-process from these fields (no
DATABASE_URL env var required).
For laptop runs against the RDS tunnel (or any local Postgres), you
can use this non-production sample (base64 of the 32-byte ASCII
string LFX-campaign-local-dev-aes-256!!):
# !!! WARNING !!!
# This is a *TEST/EXAMPLE* key for local/dev use only.
# NEVER use this in production or shared environments!
# -----------------------------------------------------
# (test/dev/example: base64-encoded 'LFX-campaign-local-dev-aes-256!!')
# Example (use printf so the *plaintext* stays exactly 32 bytes — a trailing
# newline from echo would make it 33 and break AES-256 key length):
# printf '%s' 'LFX-campaign-local-dev-aes-256!!' | base64
export CREDENTIAL_ENCRYPTION_KEY='TEZYLWNhbXBhaWduLWxvY2FsLWRldi1hZXMtMjU2ISE='Again, do not use this value in shared, staging, or production clusters. Generate a real key for those environments, for example:
openssl rand -base64 32-
PGPORT(default5432; secret keyport) — PostgreSQL port -
PGENGINE(default empty) — when set, must bepostgresorpostgresql -
PORT(default8080) — HTTP listen port (CLI flag-p) -
HOST(default*) — bind interface;*means all interfaces (CLI flag-bind) -
DEBUG(unset) — set totrueto enable debug logging (CLI flag-d) -
JWKS_URL— key set every bearer token is verified against (defaults to the in-cluster Heimdall JWKS URL). Set but not an absolute URL fails startup rather than degrading verification. -
JWT_AUDIENCE(defaultlfx-v2-campaign-service) — required audience -
JWT_ISSUER(defaultheimdall) — required issuer -
JWT_AUTH_DISABLED_MOCK_LOCAL_PRINCIPAL(default unset) — local development only: when non-empty it disables JWT verification and attributes every request to this principal, logging aWARNon every boot that sets it. Two independent guards keep it out of a deployed environment: the chart refuses to render it, and at runtime the pod fails startup rather than degrading whenever it detects it is running in a cluster — so the failure is a crash-loop, not a silently unauthenticated service. Worth knowing before debugging a rollout that will not come up. The runtime guard is a detection, not a proof: it looks forKUBERNETES_SERVICE_HOSTand the service-account directory, so a manifest applied outside the chart that both sets this variable and suppresses those two signals (an explicit emptyKUBERNETES_SERVICE_HOSTplusautomountServiceAccountToken: false) is not caught. That combination is three deliberate, visible changes rather than one env line, which is the bar the guard is built to — seeinternal/infrastructure/config/config.go. -
NATS_URL— NATS server URL (reserved for messaging; defaults to in-cluster NATS URL) -
EVENT_URL_NAT64_PREFIXES(default unset) — comma-separated RFC 6052 §2.2 network-specific NAT64 prefixes this cluster uses, e.g.2a01:4f8:1::/48. Only/32,/40,/48,/56,/64and/96are valid lengths; a malformed value or a wrong length panics at startup, deliberately, so a typo stops the pod rather than decoding at the wrong offset for its lifetime.Unset is correct for a cluster with no NAT64, and the well-known
64:ff9b::/96is decoded regardless. It cannot be discovered in-process: a network-specific prefix is carved from the operator's own global unicast space and is indistinguishable from any other public prefix. On a cluster that uses one, leaving it undeclared is a live SSRF hole — the translator, not this service, makes the IPv4 connection, so an address encoding169.254.169.254passes every check here and the fetch reaches the cloud metadata endpoint. -
REDDIT_METRICS_ENABLED(default unset, i.e. OFF) — opts a deployment IN to Reddit Ads metrics reads. Only the exact valuetrueenables them; unset or any other value (includingTRUEor a typo) fails closed, andGET .../campaigns/{campaign_id}/metricsanswers 400 "not supported for this campaign's platform" for a Reddit campaign. Off by default because no request has yet been made against a live Reddit ad account (LFXV2-3282). The request and response shapes now follow Reddit's official public OpenAPI document, but behaviour a schema cannot express — zero-activity rows, the account's attribution window — is still unconfirmed, and a read returning 200 would look authoritative to every consumer. The chart sets it to"false"(charts/lfx-v2-campaign-service/values.yaml); flip it only after the contract is verified against a live Reddit ad account. -
CAMPAIGN_JOB_RETENTION(default4320h, i.e. 180 days) — how long a terminal campaign job (succeeded,partial,failed) is kept before the retention sweeper deletes it. A Go duration string.campaign_jobsis otherwise append-only: nothing else in the service deletes a row, so without the sweeper the table grows with every brief dispatch forever, and the stuck-job recovery sweep pays for that growth on every pass. One bounded, ordered batch is deleted per pass (seeJobRepo.PruneTerminalJobs), so a large backlog drains over several passes rather than in one long transaction. Every replica runs it without leader election, exactly as the stuck-job sweeper does — overlapping passes are harmless because the delete is bounded and idempotent.Only terminal jobs are ever eligible, enumerated as an allow-list rather than as "not queued/running". A queued or running row is never deleted at any age: an old non-terminal row is a stuck job, which is precisely the record needed to investigate a dispatch that never finished. (The recovery sweeper fails those out after 15m, at which point they become terminal and their retention window starts from that transition.) Age is measured on
updated_at— when the job reached its terminal state — so a job created months ago but completed yesterday counts as recent history.The default is deliberately long because these rows are the audit trail of real ad spend. Unset, empty, unparseable (
30 daysand7dare not valid Go durations) and non-positive values all fall back to the 180-day default rather than to a short window, so a typo cannot cause early deletion — the rejected value is logged at startup. -
MICROSOFT_METRICS_ENABLED(default unset, i.e. OFF) — opts a deployment IN to Microsoft Advertising (Bing Ads) metrics reads. Only the exact valuetrueenables them; unset or any other value (includingTRUEor a typo) fails closed, andGET .../campaigns/{campaign_id}/metricsanswers 400 "not supported for this campaign's platform" for a Microsoft campaign. Off by default because the v13 Reporting contract was implemented from Microsoft's published documentation and has NOT been exercised against a live Microsoft Advertising account. Microsoft's pipeline is also unlike every other platform's — an asynchronous submit/poll/download returning a zipped CSV rather than one JSON GET — so there is more surface to be wrong about. The chart sets it to"false"(charts/lfx-v2-campaign-service/values.yaml); flip it only after the contract is verified against a live Microsoft ad account. -
LFX_FORCE_SYSTEM_ADS_ACCOUNT(default unset, i.e. OFF) — makes the LF-owned system account (system:linuxfoundation) the PRIMARY credential source when a paid-ads campaign is CREATED, so every new campaign authenticates as the marketing-ops account regardless of any per-project connection (the system row is otherwise only a fallback for projects that have connected no account of their own). Only the exact valuetrueenables it; unset or any other value (includingTRUEor a typo) leaves the default resolution.It governs creation and account discovery only. Pausing, resuming and reading metrics for a campaign that ALREADY EXISTS keep using the account it was created under, so campaigns created before the flag was turned on stay controllable while it is on — forcing them onto the system account would trip each adapter's account-provenance guard and leave a live, spending campaign the service cannot pause.
While it is on, an ad account id cannot be SAVED onto a project's connection (400); discovery is resolving the LF credential, so the ids it returns are LF-owned and persisting one would outlive the flag. Clearing a selection stays allowed, as is re-sending the id already stored — the guard refuses a CHANGED id, not a present one. This is what keeps the rollout reversible by flipping the flag back.
One consequence to plan around:
linkedin-ads,reddit-adsandmicrosoft-adsdeclareaccount_idas REQUIRED on their create payloads, so a create body cannot omit it and those three providers cannot be connected at all while the flag is on.google-ads,meta-adsand — as of LFXV2-3319 —twitter-adsare credentials-first and can still be created. (X'sfunding_instrument_idis still required; credentials-first defers the ACCOUNT choice only.) The per-endpoint status contract is indocs/api-catalog.mdunder Platform Connections.HubSpot/email is never forced (the path gates on the paid-ads channel kind). A missing or unusable system row fails the dispatch closed rather than falling through to the project connection; a MISSING one is reported as an operator-owned configuration fault (500), not as "connect your project" (404). It shares the exact-match
trueparse withREDDIT_METRICS_ENABLEDbut not its lifecycle: this flag is read ONCE, when the credentials source is constructed, so changing it requires a restart, whereasREDDIT_METRICS_ENABLEDis re-read from the environment on every metrics read. Enable per-environment via the ArgoCD overlay. Seespecs/006-force-system-ads-account.
Read-only warehouse access, used ONLY to resolve an event's past editions when
building an audience (LFXV2-2774). These are OPTIONAL as a GROUP: unless
SNOWFLAKE_ACCOUNT, SNOWFLAKE_USER and SNOWFLAKE_PRIVATE_KEY are ALL set,
the warehouse is treated as unconfigured and audience building still works —
it produces a country-only audience and records the narrower scope in the
audience's inclusion summary. A partial or unusable configuration degrades the
same way rather than failing startup, because past editions ENRICH an audience
rather than making it correct.
SNOWFLAKE_ACCOUNT— account identifierSNOWFLAKE_USER— user for key-pair authSNOWFLAKE_PRIVATE_KEY— PEM private key for key-pair authSNOWFLAKE_WAREHOUSE— optional warehouse; the account default is used when omittedSNOWFLAKE_ROLE— optional role; the user's default is used when omitted
The LF LiteLLM proxy, used ONLY to generate email copy — subject, preheader,
body and CTA — via the POST /projects/{id}/briefs/{id}/email-copy endpoint
(LFXV2-2775). Optional as a GROUP: unless BOTH AI_PROXY_URL and AI_API_KEY
are set, the endpoint returns 503 (service unavailable). The pod starts successfully
with or without these values configured.
AI_PROXY_URL— LiteLLM proxy base URL;/chat/completionsis appendedAI_API_KEY— the proxy's key, not a Bedrock or Anthropic credential (the proxy holds those), so it cannot be replayed against a model provider directlyAI_MODEL— optional, not a secret; the model id the proxy routes on. Empty selectsllm.DefaultModel.
In-cluster the two secrets come from the same ExternalSecret-managed secret as
the rest; AI_MODEL is a plain chart value.
OpenTelemetry is opt-in. Exporters default to none (no collector
required for local runs).
OTEL_SERVICE_NAME(defaultlfx-v2-campaign-service)OTEL_SERVICE_VERSION(default: build version)OTEL_EXPORTER_OTLP_PROTOCOL(defaultgrpc) —grpcorhttpOTEL_EXPORTER_OTLP_ENDPOINT— collector endpointOTEL_EXPORTER_OTLP_INSECURE(defaultfalse) — insecure whentrueOTEL_TRACES_EXPORTER(defaultnone) —otlpornoneOTEL_METRICS_EXPORTER(defaultnone) —otlpornoneOTEL_LOGS_EXPORTER(defaultnone) —otlpornoneOTEL_PROPAGATORS(defaulttracecontext,baggage) — comma-separated;jaegersupportedOTEL_TRACES_SAMPLER(defaultparentbased_traceidratiowhen unset) — sampler type (always_on,always_off,traceidratio,parentbased_*, …)OTEL_TRACES_SAMPLER_ARG(default1.0) — sampler argument; for ratio-based samplers, a value in[0.0, 1.0]
In lfx-v2-dev, Postgres is RDS. The cluster exposes it as an
ExternalName Service (lfx/rds-postgres). A plain
kubectl port-forward svc/rds-postgres … does not work
(ExternalName has no endpoints). Use a short-lived jump pod with
socat, then port-forward to that pod.
Credentials live in secret
lfx-v2-campaign-service-secrets (namespace
lfx-v2-campaign-service), keys: host, port, username,
password, dbname, engine.
# 0) Point kubectl at development (example path; adjust if needed)
export KUBECONFIG="${KUBECONFIG:-$HOME/.kube/lfx-v2-dev}"
# 1) Confirm the secret exists (do not print the password)
kubectl -n lfx-v2-campaign-service get secret \
lfx-v2-campaign-service-secrets
# 2) Read the RDS hostname from the secret (safe: host only)
RDS_HOST="$(kubectl -n lfx-v2-campaign-service get secret \
lfx-v2-campaign-service-secrets \
-o jsonpath='{.data.host}' | base64 -d)"
RDS_PORT="$(kubectl -n lfx-v2-campaign-service get secret \
lfx-v2-campaign-service-secrets \
-o jsonpath='{.data.port}' | base64 -d)"
RDS_PORT="${RDS_PORT:-5432}"
# Both must be non-empty before creating the jump pod
if [ -z "$RDS_HOST" ] || [ -z "$RDS_PORT" ]; then
echo "RDS_HOST/RDS_PORT empty — refuse to create broken tunnel" >&2
exit 1
fi
echo "tunnel target ${RDS_HOST}:${RDS_PORT}"
# 3) Start a jump pod that listens on 5432 and dials RDS.
# Do NOT use --command (replaces the socat entrypoint →
# "tcp-listen:…: executable file not found").
# Do NOT use -it (Gatekeeper blocks interactive TTYs).
# Do NOT use --rm until you are done (you need the pod alive
# for port-forward).
# Delete any prior failed pod first if needed:
# kubectl -n lfx-v2-campaign-service delete pod pg-tunnel \
# --ignore-not-found
kubectl -n lfx-v2-campaign-service delete pod pg-tunnel \
--ignore-not-found
kubectl -n lfx-v2-campaign-service run pg-tunnel \
--restart=Never --image=alpine/socat -- \
tcp-listen:5432,fork,reuseaddr \
"tcp-connect:${RDS_HOST}:${RDS_PORT}"
kubectl -n lfx-v2-campaign-service wait --for=condition=Ready \
pod/pg-tunnel --timeout=60s
# Confirm args include the real host (not tcp-connect::)
kubectl -n lfx-v2-campaign-service get pod pg-tunnel \
-o jsonpath='{.spec.containers[0].args}{"\n"}'In a second terminal (leave this running — stopping it causes
connection refused on /readyz):
export KUBECONFIG="${KUBECONFIG:-$HOME/.kube/lfx-v2-dev}"
# 4) Forward laptop:5432 -> jump pod:5432
kubectl -n lfx-v2-campaign-service port-forward \
pod/pg-tunnel 5432:5432
# Expect:
# Forwarding from 127.0.0.1:5432 -> 5432
# Later, when the service pings, you may also see:
# Handling connection for 5432In a third terminal — build, load creds, run:
export KUBECONFIG="${KUBECONFIG:-$HOME/.kube/lfx-v2-dev}"
# Always use the tunnel endpoint on the laptop, not the RDS FQDN.
# If you export PGHOST from the secret's `host` key, readyz will
# time out (laptop cannot reach private RDS directly).
export PGHOST=127.0.0.1
export PGPORT=5432
export PGUSER="$(kubectl -n lfx-v2-campaign-service get secret \
lfx-v2-campaign-service-secrets \
-o jsonpath='{.data.username}' | base64 -d)"
export PGPASSWORD="$(kubectl -n lfx-v2-campaign-service get secret \
lfx-v2-campaign-service-secrets \
-o jsonpath='{.data.password}' | base64 -d)"
export PGDATABASE="$(kubectl -n lfx-v2-campaign-service get secret \
lfx-v2-campaign-service-secrets \
-o jsonpath='{.data.dbname}' | base64 -d)"
# **Note**: This is a Local-dev sample only (see "Local / test sample key" above).
# NEVER use this in production or shared environments!
export CREDENTIAL_ENCRYPTION_KEY='TEZYLWNhbXBhaWduLWxvY2FsLWRldi1hZXMtMjU2ISE='
# Sanity-check before starting (must be 127.0.0.1, not the RDS FQDN)
echo "PGHOST=$PGHOST PGPORT=$PGPORT PGDATABASE=$PGDATABASE"
# optional: confirm the tunnel accepts TCP
nc -z 127.0.0.1 5432 && echo "tunnel port open"
make build
make run
# On startup, the log line "dependency container initialized" must
# show database=127.0.0.1:5432/<dbname> — if it shows the RDS
# hostname, stop and fix PGHOST.Smoke-check readiness (expects 200 / OK while the tunnel is up):
curl -sS -w "\nHTTP %{http_code}\n" http://127.0.0.1:8080/readyz
# expect body OK and HTTP 200
curl -sS -w "\nHTTP %{http_code}\n" http://127.0.0.1:8080/livez
# expect body OK and HTTP 200 (even if readyz would be 503)Cleanup: stop the port-forward (Ctrl-C), then delete the jump pod:
kubectl -n lfx-v2-campaign-service delete pod pg-tunnel \
--ignore-not-foundtcp-listen:…: executable file not found— used--command. Recreate withrun … -- args(no--command).- Gatekeeper TTY warning / blocked — used
-it. Omit-it. - Pod args show
tcp-connect::—RDS_HOST/RDS_PORTwere empty at create. Re-export from secret, delete pod, recreate. - Startup log shows RDS FQDN as database —
PGHOSTwas taken from secrethost. Useexport PGHOST=127.0.0.1and restart. connection refusedon 127.0.0.1:5432 — port-forward not running. Restart Terminal 2.context deadline exceededwithPGHOST=127.0.0.1— jump pod dialing wrong/empty target, or tunnel stalled. Check pod args; recreate tunnel.
- VPN / direct RDS access — if your laptop can reach the RDS
FQDN, skip the jump pod; set
PGHOST/PGPORTfrom the secrethost/portkeys andmake run. - Local Docker / Homebrew Postgres — no tunnel; use
PGHOST=127.0.0.1with local credentials. - CloudNativePG ClusterIP Service —
kubectl port-forward svc/<cnpg-rw-service> 5432:5432works without a jump pod.
See also specs/002-db-conn-check/quickstart.md for readiness /
liveness validation scenarios.
Prefer make run (above) for day-to-day Go iteration. To exercise the
Helm chart — probes, secret refs, and env wiring — build an image and
install with the local values override.
make helm-install-local installs into namespace lfx (see
HELM_NAMESPACE in the Makefile). The chart still requires secret
lfx-v2-campaign-service-secrets (keys: host, port, username,
password, dbname) in that same namespace for the required PG*
env refs. Without it the pod stays in CreateContainerConfigError.
Pick one of these before installing. Set HELM_NAMESPACE once and
pass it to the final make helm-install-local (Makefile assigns
HELM_NAMESPACE=lfx with =, so an env prefix does not override).
# Default local release namespace (Option A / C)
HELM_NS=lfx
# Option A — copy the secret from lfx-v2-dev into a local cluster.
# Use distinct source and destination contexts (adjust names to match
# `kubectl config get-contexts`). Rebuild a clean Secret so server-
# managed fields (uid, resourceVersion, …) are not re-applied.
SRC_CONTEXT="${SRC_CONTEXT:-lfx-v2-dev}"
DST_CONTEXT="${DST_CONTEXT:-kind-kind}"
kubectl --context="$SRC_CONTEXT" get secret \
lfx-v2-campaign-service-secrets \
-n lfx-v2-campaign-service -o json \
| jq '{
apiVersion: .apiVersion,
kind: .kind,
type: .type,
metadata: { name: .metadata.name, namespace: "lfx" },
data: .data
}' \
| kubectl --context="$DST_CONTEXT" apply -f -
# Option B — install into the namespace that already has the secret
HELM_NS=lfx-v2-campaign-service
# Option C — point PG* at a local database in values.local.yaml
# (override PGHOST/PGPORT/PGUSER/PGPASSWORD/PGDATABASE with `value:`
# entries instead of secretKeyRef; see values.yaml for the keys)Then:
# 1) Copy the example override (gitignored once renamed)
cp charts/lfx-v2-campaign-service/values.local.example.yaml \
charts/lfx-v2-campaign-service/values.local.yaml
# Edit values.local.yaml as needed (encryption key sample is included).
# 2) Build the image (pullPolicy: Never in the local values file)
make docker-build
# 3) Load the image into your local cluster if needed (kind example):
# kind load docker-image \
# ghcr.io/linuxfoundation/lfx-v2-campaign-service/campaign-service:latest
# 4) Install / upgrade the chart (uses HELM_NS from above)
make helm-install-local HELM_NAMESPACE="$HELM_NS"values.local.example.yaml documents the copy path and
make helm-install-local target. Uninstall with make helm-uninstall.
Common workflow targets (see the Makefile for the full list):
make all # clean → apigen → fmt → lint → test → build
make clean # remove bin/ and coverage.out
make apigen # generate API code from design/ (required before first build)
make fmt # format Go code (gofmt + simplify)
make check-fmt # verify formatting (used in CI)
make lint # run golangci-lint
make test # run tests with race detector and coverage
make build # build a local binary
make build-release # build a static release binary for Linux
make run # build and run locally (needs PG* env; see above)CI runs MegaLinter on pull requests and merge-queue entries
(.github/workflows/mega-linter.yaml, Go flavor v9.1.0). Config lives in
.mega-linter.yml. To reproduce
locally with Docker or OrbStack:
docker pull oxsecurity/megalinter-go:v9.1.0
docker run --rm \
-e DEFAULT_WORKSPACE=/tmp/lint \
-e GOTOOLCHAIN=auto \
-e MEGALINTER_CONFIG=.mega-linter.yml \
-v "$PWD:/tmp/lint:rw" \
oxsecurity/megalinter-go:v9.1.0Reports are written under megalinter-reports/ (gitignored). The first
pull is large; a full run often takes several minutes.
For a faster secrets-only check matching the CI gitleaks step:
gitleaks detect --source . --config .gitleaks.tomlNote: if this checkout is a git worktree, the containerized git_diff
linter may fail because the worktree .git path is not visible inside
the container. That does not affect GitHub Actions (normal
actions/checkout).
docs/knowledge/ is an Open Knowledge Format (OKF)
bundle — plain markdown with YAML frontmatter — that gives humans and AI
agents a structured map of this repo's architecture, Kubernetes resources,
Go packages, and feature specs. Start at
docs/knowledge/index.md.
When to update it: after merging a feature PR, changing an API endpoint, adding or modifying a Helm resource, or changing a package's responsibility.
How to update it:
- Edit the relevant existing concept file under
docs/knowledge/**, or add a new one with OKF frontmatter (type,title,description) if no existing concept covers the change. Do not regenerate withgo run ./cmd/okfgen— that tool bootstraps new subtrees and will overwrite hand-edited concept files. - Add or update the concept's
* [Title](url) - descriptionbullet in the relevantindex.md. - Add a new file
docs/knowledge/log/YYYY-MM-DD-<slug>.md(slug = ticket + short description), with a first H1 dated to match the filename followed by**Update** — <what changed and why>.One file per entry — this keeps concurrent PRs from ever editing the same log file.
Validate before pushing:
go run ./cmd/okfvalidate ./docs/knowledgeThis is the same check .github/workflows/validate-okf.yml runs in CI.
Agents are expected to do this bookkeeping automatically (see CLAUDE.md);
developers making manual changes should follow the same convention.