fix(observability): give Substrate's OTLP export a tenant and a way out - #638
Merged
Merged
Conversation
Published Helm chart
|
| Chart | agent-platform-connectivity |
| Version | 4.66.8-r72dfe194t20260924110036h00c0f9e |
| OCI reference | oci://gsoci.azurecr.io/charts/giantswarm/agent-platform-connectivity:4.66.8-r72dfe194t20260924110036h00c0f9e |
| Digest | sha256:4b6f79e6a4cf93424babef5120022e6ca795a63099c015e8dbb4370ebbea1a0b |
| Registry | public — gsoci.azurecr.io |
| Git catalog | giantswarm-test-catalog (index) |
Pull this chart
helm pull oci://gsoci.azurecr.io/charts/giantswarm/agent-platform-connectivity --version 4.66.8-r72dfe194t20260924110036h00c0f9ePosted by architect-orb · build 12661 · commit 00c0f9e · updated in place on every push
Published Helm chart
|
| Chart | agent-platform |
| Version | 4.66.8-r72dfe194t20260924110036h00c0f9e |
| OCI reference | oci://gsoci.azurecr.io/charts/giantswarm/agent-platform:4.66.8-r72dfe194t20260924110036h00c0f9e |
| Digest | sha256:2057afca72e37404efc109b8f0dc40963e0ff91abd868a87b851bcbbd9aeb65d |
| Registry | public — gsoci.azurecr.io |
| Git catalog | giantswarm-test-catalog (index) |
Pull this chart
helm pull oci://gsoci.azurecr.io/charts/giantswarm/agent-platform --version 4.66.8-r72dfe194t20260924110036h00c0f9ePosted by architect-orb · build 12662 · commit 00c0f9e · updated in place on every push
The meta chart sets substrate.podLabels to the tenant label otlp-gateway routes a headerless export by, and the connectivity chart's policies of the four exporters open the endpoint substrate.otel names instead of a fixed cluster:4317 rule on ate-api-server alone.
QuentinBisson
force-pushed
the
fix/substrate-otlp-tenant
branch
from
September 24, 2026 11:03
fbb0b57 to
00c0f9e
Compare
QuentinBisson
marked this pull request as ready for review
September 24, 2026 11:08
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On gazelle, ate-api-server, ate-controller, atelet and atenet-router export to
otlp-gateway.kube-system.svc:4317, and Tempo has no span from them (giantswarm/giantswarm#36711). No Substrate pod carriesobservability.giantswarm.io/tenant, and the chart has no headers key, so otlp-gateway drops every signal as tenant-less. Also, only the ate-api-server policy allows 4317, so the other three exports never reach the collector.Change
substrate.podLabels: {observability.giantswarm.io/tenant: giantswarm}in the meta chart. The substrate chart takes the key from1.1.0(feat(chart): podLabels on the chart's pods substrate#55, which carries feat(helm): add podLabels for the chart's pods kagent-dev/substrate#48).1.0.xignores it, so the label arrives with the re-pin's range move, not with this PR.agent-platform.substrate.otlpEgressopens the namespace and port of each enabled signal's endpoint on the four exporters' Cilium policies. It replaces ate-api-server's fixedcluster:4317rule, with a golden hold inverify-target.py.parentbased_traceidratio0.01. Spans under a kagent turn follow the turn's decision, so the ratio applies only to parentless background work.make verify-substrate-otlp.Checklist
values.yamlandvalues.schema.jsonare valid.