feat: Add Buildkite integration

[immz4]

Implements #2360

This PR implements integration for the Buildkite, which allows users to:

• Create new builds
• Trigger logic when build gets finished

Integration requires following:

• Buildkite API key with read_organizations, read_user, read_pipelines, read_builds and write_builds permissions
• Manual webhook setup (unfortunately Buildkite API does not provide methods for managing webhooks, like Semaphore) for build.finished event

API keys are created in Personal Settings > API Access Tokens

Webhook is created in Organization Settings > Notification Services with following parameters:

• Webhook URL - https://<backend_address>/api/v1/integrations/<integration_id>/webhook (URL is provided during the setup)
• Token - use generated string
• Events - build.finished
• Pipelines - All Pipelines

Limitations:

• Currently "Trigger Build" actions relies on polling build result every 30s

Video: https://youtu.be/YOXcCU9rAS0

[AleksandarCole]

@immz4 thanks for the submission and a clear overview!

As you already mentioned - creating webhooks manually is not ideal. Even though they don't support API endpoint for the organization notification webhooks - does it make sense to use pipeline webhooks? If I understand correctly pipeline on Buildkite is equivalent of project on Semaphore, but I might be wrong.

[immz4]

@immz4 thanks for the submission and a clear overview!

As you already mentioned - creating webhooks manually is not ideal. Even though they don't support API endpoint for the organization notification webhooks - does it make sense to use pipeline webhooks? If I understand correctly pipeline on Buildkite is equivalent of project on Semaphore, but I might be wrong.

@AleksandarCole Unfortunately these are incoming webhooks. To quote GraphQL API doc:

Create a webhook on the repository for a pipeline. This mutation creates a webhook on the pipeline's source repository (e.g., GitHub) that will trigger builds when code is pushed. Requires a GitHub App integration to be configured for the organization. Returns an error if a webhook already exists for the repository.

So, they work like this: GitHub > Buildkite

[AleksandarCole]

@immz4 understood. Can we then work on improving the UX of configuring this? I did not manage to connect it based on the instructions in the UI. Even after reviewing the video I'm not clear how to connect this.

We can ask the user to input the link to their organization (eg. https://buildkite.com/superplane) and then based on that generate links like https://buildkite.com/organizations/superplane/services/webhook/new and https://buildkite.com/user/api-access-tokens/new to point them to the pages instead of them having to figure it out based on descriptions.

We also need to generate webhook url for them, I am unsure what this means Webhook URL is constructed in following way - https://<Backend URL>/api/v1/integrations/<Integration ID>/webhook.

[immz4]

@AleksandarCole addressed UX, added setup helpers

Integration setup flow:

---

---

"Continue" button leads to Personal Settings > API Access Token

---

"Continue" button opens "Add Webhook Notification" setup

[immz4]

I can also record new video to demonstrate UX changes.

[AleksandarCole]

Hey @immz4 this is much better, thanks! I've managed to run it - great direction, I love that you included those env vars + metadata.

Have few changes that we need to make and one problem I came across:

Environment variable broken

When I add environment variable I cannot save the component:

Wrong state on fail

If the pipeline fails state should not be Error but Fail instead, since Fail is a valid result that emits event further. Error is reserved for the cases when something goes wrong (eg. didn't manage to start the build) and it should not emit an event further through any of the output channels. Check how Semaphore component handles this.

Wrong component icon

Icons render as puzzle (because they are not using BuildKite icon)

Build metadata not set

Even though I've configured build metadata it doesn't seem to be set

Update details tab

In the details tab, let's have two timestamps at the top (Started at, and Completed at), and Build URL, feel free to delete the rest.

From the trigger component detail tab, let's also have timestamp at the top (single one in this case) and remove Build State (or result, having both is redundant), Build Number, Project, Organization (those are empty)

Integration Setup UX

The organization URL is prefilled with user email (which is very strange) and also API token is prefilled with something also. Let's make sure those are empty and that user needs to add them.

The instructions for creating the webhook appear only after I save the integration for the first time (with API token) - any way for them to be there from the start - or appear once user enters the url?

Also, would be good if we had a link in the description for API token that immediately leads you to https://buildkite.com/user/api-access-tokens as it's a fixed url (same for everyone).

[immz4]

Hey, @AleksandarCole, regarding your comments

The organization URL is prefilled with user email (which is very strange) and also API token is prefilled with something also. Let's make sure those are empty and that user needs to add them.

That is indeed very strange, since I do not have this problem.

1. Do you have the same issue with Semaphore integration setup?
2. What browser and password manager do you use? They sometimes can autofill in the wrong fields.

[cursor]

Webhook signature verification accepts future timestamps

Low Severity

The replay protection check time.Since(sigTime) > MaxReplayAge only rejects timestamps older than 5 minutes. For a future timestamp, time.Since returns a negative duration, which is always less than MaxReplayAge, so the check passes. This means a signature with a far-future timestamp remains valid indefinitely, weakening the replay protection window.

 

[lucaspin]

@immz4 after looking a bit deeper on this one, it seems clear to me that the webhook setup should be done as part of the trigger setup itself, and not the integration setup. A few considerations that led me to this conclusion:

• What if we want to add an onAgent trigger next? We would either have to (1) tell to user to update the webhook set up as part of the integration, or (2) make the user create the webhook with all the possible event types in the first place. (1) is confusing, and (2) would lead us to receive events for everything even if the user hasn't even create a trigger.
• Also, what if we want to have a trigger for test engine alarms or new packages? Here, it's not OK to be part of the integration, because those webhooks are scoped to test suite and package registry, respectively. So, we would end up with having to set up webhook during the integration setup, and also during trigger setup, for some triggers. That's inconsistent, and confusing.

---

For reference on how to handle webhook setup as part of trigger setup, take a look at the dockerhub.onImagePush trigger.

[cursor]

Pipeline filter silently bypassed on missing slug

Medium Severity

The pipeline filter uses a triple-nested optional type-assertion chain. If payload["pipeline"] is absent or present but lacks a slug key, both inner assertions fail silently and the filter is skipped entirely — the event is accepted and emitted regardless of which pipeline sent it. A webhook payload that doesn't include pipeline.slug (e.g., from a different event shape or a misconfigured Buildkite webhook) will therefore bypass the configured pipeline filter and trigger unintended workflow executions.

 

[immz4]

@lucaspin Moved webhook setup to trigger.

Current flow:

User needs to select the pipeline and save the trigger in order to get webhook url and token

---

After that user can click a button and copy over params to Buildkite

[cursor]

Slug regex rejects valid single-character org slugs

Low Severity

The slugPattern regex ^[a-zA-Z0-9][a-zA-Z0-9_-]*[a-zA-Z0-9]$ requires at least two characters because it demands both a start and end character class match. A valid single-character org slug (e.g., "a") would be rejected with an "invalid organization format" error, even though Buildkite permits it.

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}

[cursor]

Pipeline filter silently bypassed on missing slug

Low Severity

The pipeline filter in HandleWebhook is nested inside two type assertions. If payload["pipeline"] is absent or not a map[string]any, or if pipeline["slug"] is not a string, both assertions fail silently — the filter is completely bypassed and the event proceeds to trigger a workflow execution regardless of which pipeline sent it. The same silent-bypass pattern applies to the branch filter. Any build.finished event that lacks expected fields passes unfiltered.

 

[cursor]

Single-character org slugs rejected as invalid

Low Severity

The slug validation regex ^[a-zA-Z0-9][a-zA-Z0-9_-]*[a-zA-Z0-9]$ requires a minimum of two characters (one anchored at each end with zero-or-more in between), so any single-character org slug fails validation and returns an error. This is also inconsistent: the same single-character slug succeeds if the user provides it as a full URL (e.g., https://buildkite.com/a) because the URL pattern matches first, bypassing the slug pattern entirely.

 

[immz4]

@lucaspin @AleksandarCole Hi, any updates on PR merge?

[AleksandarCole]

Hey @immz4, it looks good, our internal team will need to do some final polish and updates before the merge, but I think this contract is completed successfully 👍
Thank you for your contribution and work invested 🙌