Skip to content

Commit f904b2f

Browse files
committed
feat(orchestrator): build extension profiles from configuration
## Summary ### Why? Which implementation of each extension a queue resolved to was hardcoded: the build runner was the fake one for every queue regardless of what was available, and the change provider was a global all-or-nothing environment gate. A deployment could not run one queue against a real provider next to one running entirely on fakes, which is what a stack serving both a test environment and a live repository needs. ### What? `PROFILES_CONFIG_PATH` names a YAML file selecting the change provider, build runner, and conflict analyzer per queue. Each extension is independently optional, so a queue that differs only in its analyzer says only that. `kind` is an open string rather than a closed schema, so supporting a new provider is a new value and an implementation behind it, not a change to the file's shape. The file holds no secret: each integration names the environment variable carrying its credential. The `pathoverlap` analyzer takes a `by` granularity — `file`, or `directory` to coarsen a queue's conflicts to a shared parent. It defaults to `file`, the narrower of the two: a default that widened what conflicts would serialize a queue more than its configuration asked for. With no config file the built-in example topology applies, reproducing the previous behavior exactly — including the per-queue analyzers the E2E suite depends on, and a routing change provider that still falls back to the fake when no token is set. That is what keeps the existing suite meaningful as a regression gate. Extensions are reused across queues configured alike. This is load-bearing for the build runner: the build and buildsignal controllers look it up separately and the fake holds a build's outcome in memory, so two instances would lose the result between triggering a build and polling it. ## Test Plan ✅ `bazel test //service/submitqueue/orchestrator/server:go_default_test` — pins the built-in topology against what the E2E suite expects, covers per-extension inheritance, provider defaults, every validation rejection, that queues configured alike share one build runner, that each queue's analyzer behaves as configured, and that a missing token fails at startup rather than mid-merge. ✅ Added with the `by` granularity: it defaults to `file`, `directory` is accepted, and an unrecognized value is rejected at load rather than at the first queue that resolves it.
1 parent 9e5bb3b commit f904b2f

5 files changed

Lines changed: 1406 additions & 264 deletions

File tree

service/submitqueue/orchestrator/server/BUILD.bazel

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,13 +8,15 @@ exports_files(
88
go_library(
99
name = "orchestrator_lib",
1010
srcs = [
11+
"config.go",
1112
"main.go",
1213
"profiles.go",
1314
],
1415
importpath = "github.com/uber/submitqueue/service/submitqueue/orchestrator/server",
1516
visibility = ["//visibility:private"],
1617
deps = [
1718
"//api/submitqueue/orchestrator/protopb:go_default_library",
19+
"//platform/buildkite:go_default_library",
1820
"//platform/errs/generic:go_default_library",
1921
"//platform/errs/mysql:go_default_library",
2022
"//platform/extension/consumergate:go_default_library",
@@ -23,12 +25,15 @@ go_library(
2325
"//platform/extension/counter:go_default_library",
2426
"//platform/extension/counter/mysql:go_default_library",
2527
"//platform/extension/messagequeue/mysql:go_default_library",
28+
"//platform/githubactions:go_default_library",
2629
"//platform/http:go_default_library",
2730
"//platform/pipeline:go_default_library",
2831
"//submitqueue/core/changeset:go_default_library",
2932
"//submitqueue/entity:go_default_library",
3033
"//submitqueue/extension/buildrunner:go_default_library",
34+
"//submitqueue/extension/buildrunner/buildkite:go_default_library",
3135
"//submitqueue/extension/buildrunner/fake:go_default_library",
36+
"//submitqueue/extension/buildrunner/githubactions:go_default_library",
3237
"//submitqueue/extension/changeprovider:go_default_library",
3338
"//submitqueue/extension/changeprovider/fake:go_default_library",
3439
"//submitqueue/extension/changeprovider/github:go_default_library",
@@ -54,6 +59,7 @@ go_library(
5459
"//submitqueue/orchestrator:go_default_library",
5560
"@com_github_go_sql_driver_mysql//:go_default_library",
5661
"@com_github_uber_go_tally//:go_default_library",
62+
"@in_gopkg_yaml_v3//:go_default_library",
5763
"@org_golang_google_grpc//:go_default_library",
5864
"@org_golang_google_grpc//reflection:go_default_library",
5965
"@org_golang_x_oauth2//:go_default_library",
@@ -91,9 +97,13 @@ filegroup(
9197

9298
go_test(
9399
name = "go_default_test",
94-
srcs = ["profiles_test.go"],
100+
srcs = [
101+
"config_test.go",
102+
"profiles_test.go",
103+
],
95104
embed = [":orchestrator_lib"], # keep
96105
deps = [
106+
"//submitqueue/entity:go_default_library",
97107
"//submitqueue/extension/buildrunner:go_default_library",
98108
"//submitqueue/extension/changeprovider:go_default_library",
99109
"//submitqueue/extension/conflict:go_default_library",
@@ -102,5 +112,7 @@ go_test(
102112
"//submitqueue/extension/storage:go_default_library",
103113
"@com_github_stretchr_testify//assert:go_default_library",
104114
"@com_github_stretchr_testify//require:go_default_library",
115+
"@com_github_uber_go_tally//:go_default_library",
116+
"@org_uber_go_zap//zaptest:go_default_library",
105117
],
106118
)

0 commit comments

Comments
 (0)