Commit 2544f02
authored
feat(storage): shard every table by queue (#543)
## Summary
### Why?
An audit of all 22 `schema/*.sql` files found five tables that are not
shardable by queue: their primary key does not lead with the queue, so
one queue's rows stay reachable through another queue's binding and the
tables cannot be split across shards. The five are
`speculation_path_set`, `counter`, `request_summary`, `request_log`, and
`change_uri_request_mapping`.
Three of those — the gateway read-model tables — were declared
permanently unshardable by `submitqueue/extension/storage/storage.go`
and the schema README, on the grounds that their lookups "start from
identifiers that arrive without queue context". This change removes that
exception at its source rather than working around it: the gateway read
APIs now require the queue alongside the sqid or change URI, so the
queue is always available at the call site and no identifier ever has to
be parsed to recover one.
The five platform `messagequeue` tables are deliberately out of scope.
They are a message-queue backend keyed by `(consumer_group, topic,
partition_key)`, not a domain table set, and sharding them is a separate
problem (a global `AUTO_INCREMENT` offset, and a subscriber-heartbeat
table whose fair-leasing logic genuinely needs the whole live subscriber
set).
### What?
All 17 in-scope tables now lead their primary key with the queue.
**Independent re-keys.** `speculation_path_set` moves to `(queue, head)`
— a batch ID is unique only within its queue, so the head alone was
never a safe key. `counter` moves to `(queue, domain)` and gains the
standard extension `Config`/`Factory` shape; per the extension contract
the factory *implementations* live in the three service `main.go` files,
not under `platform/extension/`.
**Minted identifiers are unchanged.** Counter domains simplify to bare
`"request"` / `"batch"`, but every emitted ID keeps its exact current
format: `{queue}/{seq}`, `{queue}/batch/{n}`, and stovepipe's
`request/{queue}/{seq}`. Stovepipe previously reused its counter domain
*as* the ID prefix, which would have silently turned `request/queueA/7`
into `request/7`; the two are now written independently so they cannot
drift into each other. Note that re-keying `counter` restarts every
sequence at 1, so its table must be recreated in the same cutover as
`request`, `batch`, `request_summary`, and `request_log` — never on its
own, or freshly minted sqids will collide with surviving rows.
**Queue required in the gateway contract.** `CancelRequest`,
`GetRequestSummaryByIDRequest`, `GetRequestSummaryByChangeURIRequest`,
`GetRequestHistoryByIDRequest`, and
`GetRequestHistoryByChangeURIRequest` each gain a `queue` field,
validated on entry through the existing `validateQueueIdentifier`. Two
behaviour changes follow. The change-URI lookups are now scoped to one
queue, so the same URI landed into several queues needs one call per
queue. And `Cancel` no longer overrides the caller's queue with the
stored one — a mismatched queue yields `NotFound`, since a sqid is
simply not resolvable outside its own queue.
**Read-model tables re-keyed and folded in.** `request_summary`,
`request_log`, and `change_uri_request_mapping` move to queue-leading
keys and join the `Storage` aggregate; the `SetGlobalStores` seam and
the "deliberately not part of this aggregate" carve-out are deleted. The
queue travels as an explicit field on `entity.RequestLog` and
`entity.RequestURI`, stamped by producers that already know it — no ID
parsing is introduced anywhere.
**Enforcement.** A new `//tool/linter/queueshard` walks the schema
directories and fails if any primary key does not lead with the queue,
or if any secondary index does not — a non-queue-leading index would
reintroduce exactly the cross-queue access path the primary key just
closed. It is wired into `make lint`.
Schema changes are clean recreates, not online migrations, on the schema
README's statement that these tables are created empty at rollout and
never backfilled.
## Test Plan
- ✅ `make test` — 93/93 pass, including the new linter's own tests
- ✅ `bazel test //test/integration/...` — 8/8 suites pass against real
MySQL, including new cross-queue isolation coverage in the counter,
storage, request-URI and request-log contract suites
- ✅ `bazel test //test/e2e/...` — both suites pass against the full
Docker Compose stack (land → landed, plus status/history/cancel/list
through the queue-scoped APIs)
- ✅ `make lint` — fmt, license headers, and `queueshard` ("All 17 tables
are shardable by queue")
- ✅ `make fmt` / `make gazelle` / `make mocks` / `make tidy` — no drift
The e2e and image-building integration targets need
`--sandbox_writable_path=$HOME/.docker` when run locally; `make
e2e-test` does not pass it, which is a pre-existing local-only issue
unrelated to this change.1 parent 9467629 commit 2544f02
82 files changed
Lines changed: 1703 additions & 640 deletions
File tree
- api/submitqueue/gateway
- protopb
- proto
- doc/rfc/submitqueue
- platform/extension/counter
- mock
- mysql
- schema
- service
- stovepipe/server
- submitqueue
- gateway/server
- mapper
- orchestrator/server
- stovepipe/controller
- submitqueue
- core/request
- entity
- extension/storage
- mock
- mysql
- schema
- gateway/controller
- log
- orchestrator
- controller
- batch
- mergeconflictsignal
- start
- test
- e2e/submitqueue
- integration
- extension/counter
- mysql
- submitqueue
- extension/storage
- mysql
- gateway
- tool/linter/queueshard
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
172 | 172 | | |
173 | 173 | | |
174 | 174 | | |
175 | | - | |
| 175 | + | |
176 | 176 | | |
177 | 177 | | |
178 | 178 | | |
| |||
182 | 182 | | |
183 | 183 | | |
184 | 184 | | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
185 | 188 | | |
186 | 189 | | |
187 | 190 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
74 | 74 | | |
75 | 75 | | |
76 | 76 | | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
77 | 80 | | |
78 | 81 | | |
79 | 82 | | |
| |||
106 | 109 | | |
107 | 110 | | |
108 | 111 | | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
109 | 115 | | |
110 | 116 | | |
111 | 117 | | |
| |||
118 | 124 | | |
119 | 125 | | |
120 | 126 | | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
121 | 130 | | |
122 | 131 | | |
123 | 132 | | |
| |||
152 | 161 | | |
153 | 162 | | |
154 | 163 | | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
155 | 167 | | |
156 | 168 | | |
157 | 169 | | |
| |||
176 | 188 | | |
177 | 189 | | |
178 | 190 | | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
179 | 194 | | |
180 | 195 | | |
181 | 196 | | |
| |||
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
0 commit comments