Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion SAFETY.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ The invariant registry (invariant IDs referenced below) lives in
| --- | --- | --- | --- |
| `pkg/dbconn` — pool defaults, terminate-blockers, retries, RDS TLS; advisory table lock planned | ✅ core | exists; advisory table lock planned | LK-2 primitives; LK-1 planned |
| `pkg/preflight` — precondition verifier, refusals | ✅ core | exists (Phase 1: table-size guard); grows through Phase 2 | ST-6, RF-1..RF-5 |
| `pkg/executor` — bounded optimistic attempt; native concurrent index build with invalid-index recovery; remaining native idioms at Phase 3 | ✅ core | exists (Phase 1: attempt-under-budget; Phase 3.1: concurrent index build) | LK-2 (attempt bound + the CONCURRENTLY wait-policy exception) |
| `pkg/executor` — bounded optimistic attempt; native concurrent index build with invalid-index recovery; native sequence executor for the safer idioms | ✅ core | exists (Phase 1: attempt-under-budget; Phase 3.1: concurrent index build; Phase 3.2: sequence executor) | LK-2 (attempt bound + the CONCURRENTLY wait-policy exception) |
| `pkg/checksum` — chunk verifier, continuous checker, repair | ✅ core | planned (Phase 5) | CO-1, CO-2, CO-3 |
| `pkg/copier` — shadow-table chunked copy | ✅ core | planned (Phase 4) | CO-4, LK-3 |
| `pkg/applier` — change apply, buffer, flush scheduling | ✅ core | planned (Phase 6) | CO-4, CO-5, CO-6, LK-3 |
Expand Down
7 changes: 4 additions & 3 deletions docs/high-level-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -239,9 +239,10 @@ Two principles govern this:
an **explicit confirmation** (typed acknowledgement, not a bare `-y`), and the override is
logged. Force is an escape hatch, not a convenience.

Today the classifier constructs safer sequences and `diff` / `migrate --dry-run` render them;
default `migrate` still uses the bounded optimistic Phase 1 path. Phase 3 adds substitution and
execution of classifier-produced SQL.
Today the classifier constructs safer sequences, `diff` / `migrate --dry-run` render them, and
the library's sequence executor runs them under the autocommit-each-step contract; default
`migrate` still uses the bounded optimistic Phase 1 path. Phase 3's remaining work is the
substitution wiring that routes the classified sequences into `migrate`.

In non-interactive contexts (CI), advisory mode is a natural gate: the engine prints the
recommended rewrites and exits non-zero if a submitted statement would need a riskier path than
Expand Down
17 changes: 12 additions & 5 deletions docs/invariants.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,10 +139,15 @@ The cutover swap is the only `ACCESS EXCLUSIVE` acquisition in the happy path, a
strong-lock acquisition (swap, catalog flips, trigger install in fallback mode) runs under
`lock_timeout` + bounded retry/backoff so the engine never sits at the head of the lock queue
(mysql-vs-postgresql § the lock queue).
**Exception policy required:** `CREATE INDEX CONCURRENTLY` (and `REINDEX CONCURRENTLY`,
`VALIDATE CONSTRAINT`) wait on other transactions via lock waits that a naive `lock_timeout`
cancels — leaving an `INVALID` index. These statements get their own wait policy rather than the
blanket timeout. *Enforced:* every DDL execution path in the native and copy-and-swap executors.
**Exception policy required:** `CREATE INDEX CONCURRENTLY` and `REINDEX CONCURRENTLY` wait on
other transactions via lock waits that a naive `lock_timeout` cancels — leaving an `INVALID`
index — so they get their own wait policy (no per-lock timeout, one overall statement deadline)
rather than the blanket timeout. `VALIDATE CONSTRAINT` is different in kind: its cancellation is
transactionally clean (the constraint simply stays `NOT VALID`; no debris), so the sequence
executor's validate class deliberately keeps a bounded per-lock timeout — queueing behind a
conflicting lock holder must not stall a sequence for the whole scan budget — while the scan
itself runs under its own generous overall budget. *Enforced:* every DDL execution path in the
native and copy-and-swap executors.
*Source:* [design-principles](design-principles.md#correctness-and-safety), mysql-vs-postgresql;
CIC exception from the validation review.

Expand Down Expand Up @@ -238,7 +243,9 @@ which enforces exactly one statement through the real grammar — and refuses, b
executes, any statement whose target table does not match the preflight proof it was handed.
A proof for one table can never smuggle SQL against another, and a multi-statement string can
never reach the database through the executor (pgx's simple protocol would happily run all of
it). *Enforced:* `pkg/executor` (`ExecuteNative`), `pkg/statement` (proof construction).
it). *Enforced:* `pkg/executor` (`ExecuteNative`; `RunSequence` admission re-proves every step's
target against the preflight proof before the first step executes), `pkg/statement` (proof
construction).
*Source:* adversarial review of the optimistic front door.

## Refusals and preflight (RF)
Expand Down
12 changes: 9 additions & 3 deletions docs/low-level-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -340,8 +340,15 @@ Classification belongs to `pkg/planner`; `pkg/statement` supplies typed operatio
| `migrate` (default) | Run the Phase 1 statement gate, preflight, and bounded optimistic native attempt. It does not yet execute classifier-produced safer SQL. |
| `migrate --force` (planned Phase 3) | Run each statement **exactly as submitted**, bypassing the safe rewrite. Gated — see below. |

The classifier constructs `CREATE INDEX CONCURRENTLY` and other safer sequences today, but only
`diff` and `migrate --dry-run` render them. Phase 3 makes the classified route drive execution.
The classifier constructs `CREATE INDEX CONCURRENTLY` and other safer sequences today, and the
library executes the multi-step idiom families — `pkg/executor`'s sequence executor runs a safer
sequence under the autocommit-each-step contract (brief steps bounded like an optimistic
attempt, the validation scan and concurrent builds under their own budgets). The one-step
`CONCURRENTLY` rewrites the classifier also emits (`DROP INDEX`, `REINDEX`,
`DETACH PARTITION`) are not yet driven: the sequence executor refuses them typed, because a
cancelled wait leaves recovery states it does not yet own. The CLI front door does not yet
route to it: `diff` and `migrate --dry-run` render the sequences, and Phase 3's substitution
work wires the classified route into execution.

### The `--force` gate

Expand Down Expand Up @@ -800,7 +807,6 @@ roughly in order:
each executor outcome gaining a stable string code in the report contracts, the same
treatment `pkg/lint` gave its findings, so orchestrators branch on one vocabulary,
- execute classifier-produced safer sequences through the routed native path,
- the remaining native idioms (`NOT VALID`+`VALIDATE`, `ADD PK USING INDEX`, fast-default),
- bound lock acquisition with timeout and retry for the blocking idioms,
- substitution by default, the guarded `--force` escape hatch, and progress reporting
(`pg_stat_progress_create_index` by the build's backend PID, which the executor already
Expand Down
10 changes: 10 additions & 0 deletions pkg/executor/optimistic.go
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,16 @@ func (b Budget) validate() error {
if b.StatementTimeout < minBudget {
return fmt.Errorf("statement budget must be at least %s, got %s", minBudget, b.StatementTimeout)
}
// Both settings are int32-millisecond server GUCs sent raw via
// SET LOCAL: a value beyond the server ceiling would be rejected
// mid-attempt as an out-of-range setting — an operational error where
// a budget defect decidable here should refuse at admission.
if b.LockTimeout > maxOverallBudget {
return fmt.Errorf("lock budget must be at most %s, got %s", maxOverallBudget, b.LockTimeout)
}
if b.StatementTimeout > maxOverallBudget {
return fmt.Errorf("statement budget must be at most %s, got %s", maxOverallBudget, b.StatementTimeout)
}
return nil
}

Expand Down
Loading
Loading