Repository navigation
builds: fix OTel blocked status and restore missing group columns - #111
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Two fixes for the OTel-backed Builds tab (live when CI_DATA_SOURCE=otel): 1. Statuses showed "blocked" instead of "passed". The vLLM CI pipeline ends at a manual gate, and the OTel build span's terminal state stays "blocked" even after the build is unblocked and passes. Map blocked to passed for display so the status column matches the warehouse (verified: every blocked OTel build is "passed" in Databricks). 2. Group columns went missing. OTel only emits spans for jobs that execute, so the extended-matrix jobs gated behind the manual unblock step never produce spans and their group columns vanished. Read the full job roster (including blocked jobs) from the Buildkite REST API Get-a-build endpoint instead of otel_spans, falling back to OTel spans if the roster is empty so the table degrades gracefully instead of going blank. Co-authored-by: Kimi Code <noreply@anthropic.com> Signed-off-by: khluu <khluu000@gmail.com>
khluu
force-pushed
the
otel-builds-state-fix
branch
from
September 3, 2026 04:41
5224069 to
44f52ec
Compare
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #105 fixing two regressions in the OTel-backed Builds tab (live now that
CI_DATA_SOURCE=otel):1. Statuses showed "blocked" instead of "passed"
The vLLM CI pipeline ends at a manual gate. When the automatic jobs finish, the build sits in
blockedstate awaiting an unblock. The OTelbuildkite.buildspan is emitted once withblockedand is not updated to the resolved state. Fivetran/Databricks re-syncs the record from the REST API, so it shows the finalpassed.Fix: map
blockedtopassedfor display in the builds query and duration chart.Verified against the production
otel_spanstable: every OTelblockedbuild (including ones with failed-but-retried or gated-matrix failures, e.g. #86349 with 18) ispassedin Databricks. Sample of the fixed query:2. Group columns went missing
OTel only emits spans for jobs that execute. The ~250-400 extended-matrix jobs gated behind the manual unblock step never run, so they produce no spans and their group columns vanished. The complete job roster only exists in the Buildkite API.
Fix: the groups/jobs endpoints now read the full roster from the Buildkite REST API (
Get a build), which includes blocked jobs. Falls back to OTel spans if the roster returns empty (e.g. token misconfigured), so the table degrades gracefully instead of going blank.Not a duplicate
Checked open PRs (
gh pr list --search "otel",--search "blocked") — only #98 (OpenAPI docs) and #73 (Docker traces), both unrelated.Test plan
npx tsc --noEmit— cleannpx eslinton touched files — cleannpm test— 56/56 passnpm run build— succeedsotel_spanstable (query output above).Not yet verified
The REST roster call needs
BUILDKITE_API_TOKEN, which is a hidden Vercel secret and not available locally, so I could not exercise that path end-to-end before opening this. The endpoint follows the documentedGet a buildshape, and there is a graceful fallback to OTel spans. I want to verify the columns render on a live deployment before relying on this — preview deployments are SSO-protected, so I will confirm on production right after merge (the fallback makes this low-risk).AI assistance
Developed with AI assistance (Kimi Code). I reviewed the diff and ran the checks above.