Add deployment-managed core migration primitives - #2437
Conversation
🦋 Changeset detectedLatest commit: f5403a7 The changes in this PR will be included in the next version bump. This PR includes changesets to release 17 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
docs | e450006 | Aug 18 2026, 01:17 PM |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
emdash-demo-cache | f5403a7 | Aug 18 2026, 02:25 PM |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
emdash-demo-do | f5403a7 | Aug 18 2026, 02:29 PM |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
emdash-playground | 0489cd4 | Aug 13 2026, 11:27 AM |
Scope checkThis PR changes 3,358 lines across 51 files. Large PRs are harder to review and more likely to be closed without review. If this scope is intentional, no action needed. A maintainer will review it. If not, please consider splitting this into smaller PRs. See CONTRIBUTING.md for contribution guidelines. |
@emdash-cms/admin
@emdash-cms/auth
@emdash-cms/auth-atproto
@emdash-cms/blocks
@emdash-cms/cloudflare
@emdash-cms/contentful-to-portable-text
emdash
create-emdash
@emdash-cms/gutenberg-to-portable-text
@emdash-cms/plugin-cli
@emdash-cms/plugin-types
@emdash-cms/registry-client
@emdash-cms/registry-lexicons
@emdash-cms/registry-verification
@emdash-cms/sandbox-workerd
@emdash-cms/x402
@emdash-cms/plugin-ai-moderation
@emdash-cms/plugin-atproto
@emdash-cms/plugin-audit-log
@emdash-cms/plugin-color
@emdash-cms/plugin-embeds
@emdash-cms/plugin-field-kit
@emdash-cms/plugin-forms
@emdash-cms/plugin-webhook-notifier
commit: |
There was a problem hiding this comment.
Pull request overview
Adds the core building blocks for deployment-managed core database migrations in packages/core, including a runtime policy (auto/check/manual), a validated secret-free migration manifest written at build/sync time, direct migration executors for SQLite/libSQL/Postgres, and public exports (identity + exact status) intended for deployment tooling.
Changes:
- Introduces runtime migration modes and enforces them during runtime initialization + middleware handling.
- Adds migration manifest primitives (identity, manifest schema/validation, manifest writer, config loader, integration metadata) and writes
.emdash/migrations.jsonduringastro build/astro sync. - Adds direct migration executors for SQLite/libSQL/Postgres plus a new exact migration status API (
knownApplied/pending/unknownApplied) with broad test coverage.
Reviewed changes
Copilot reviewed 50 out of 51 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| packages/core/tsdown.config.ts | Adds new migration-related entrypoints to the build. |
| packages/core/tests/utils/test-db.ts | Exposes exact migration status helper for dialect tests. |
| packages/core/tests/unit/migrations/manifest.test.ts | Unit coverage for manifest build + validation (schema, secrets, entrypoints). |
| packages/core/tests/unit/migrations/manifest-writer.test.ts | Unit coverage for atomic manifest writes and failure cleanup. |
| packages/core/tests/unit/migrations/identity.test.ts | Unit coverage for identity fingerprinting and immutability. |
| packages/core/tests/unit/migrations/direct-executor.test.ts | Unit coverage for direct executor behavior, lifecycle, and failure modes. |
| packages/core/tests/unit/migrations/config-loader.test.ts | Unit coverage for config discovery/evaluation + metadata extraction + i18n normalization. |
| packages/core/tests/unit/i18n/normalize.test.ts | Unit coverage for Astro i18n normalization helper. |
| packages/core/tests/unit/db/migration-executors.test.ts | Unit coverage for sqlite/libsql/postgres migration executor modules. |
| packages/core/tests/unit/db/migration-adapters.test.ts | Unit coverage for adapter migration descriptors and secret separation. |
| packages/core/tests/unit/database/migration-policy.test.ts | Unit coverage for migration config normalization + runtime enforcement query counts. |
| packages/core/tests/unit/astro/setup-dev-bypass.test.ts | Ensures dev-bypass/setup routes don’t run migrations directly. |
| packages/core/tests/unit/astro/middleware-migration-policy.test.ts | Ensures middleware returns generic 503 behavior on pending migrations/manual mode. |
| packages/core/tests/unit/astro/integration/migration-policy.test.ts | Ensures integration rejects invalid migration policy modes. |
| packages/core/tests/unit/astro/integration/migration-manifest.test.ts | Ensures build/sync write secret-free manifests and metadata is attached correctly. |
| packages/core/tests/integration/runtime/create.test.ts | Ensures check mode doesn’t trigger migration-failure backoff. |
| packages/core/tests/integration/database/migrations.test.ts | Adds integration coverage for exact migration status behavior + query counts. |
| packages/core/tests/integration/database/dialect-compat.test.ts | Ensures exact migration status works across dialects. |
| packages/core/src/virtual-modules.d.ts | Extends virtual config typing to include migrations config. |
| packages/core/src/migrations/target.ts | Adds safe target fingerprinting + env var resolution helper. |
| packages/core/src/migrations/protocol.ts | Defines executor protocol types (request/target/report/factory). |
| packages/core/src/migrations/manifest.ts | Adds manifest v1 schema, secret-free validation, and serialization. |
| packages/core/src/migrations/manifest-writer.ts | Writes validated manifest atomically to .emdash/migrations.json. |
| packages/core/src/migrations/manifest-builder.ts | Builds a manifest from identity + i18n + adapter migration metadata. |
| packages/core/src/migrations/integration-metadata.ts | Attaches secret-free migration metadata to the Astro integration (symbol-keyed). |
| packages/core/src/migrations/index.ts | Exports migration identity, protocol, and direct executor APIs. |
| packages/core/src/migrations/identity.ts | Adds core migration identity + fingerprint generation. |
| packages/core/src/migrations/direct-executor.ts | Implements single-use direct executor (check/apply) with i18n scoping. |
| packages/core/src/migrations/config-loader.ts | Adds config-file discovery and safe evaluation fallback to build manifests from config. |
| packages/core/src/index.ts | Exports exact migration status + MIGRATION_NAMES/types from the public entry. |
| packages/core/src/i18n/normalize.ts | Adds normalization of Astro i18n input into EmDash’s I18nConfig. |
| packages/core/src/emdash-runtime.ts | Enforces runtime migration policy during DB init; avoids backoff for pending-migration errors. |
| packages/core/src/db/sqlite-migrations.ts | Adds deployment executor module for SQLite migrations. |
| packages/core/src/db/postgres-migrations.ts | Adds deployment executor module for Postgres migrations. |
| packages/core/src/db/libsql-migrations.ts | Adds deployment executor module for libSQL migrations. |
| packages/core/src/db/index.ts | Exposes exact migration status and MIGRATION_NAMES via db exports. |
| packages/core/src/db/adapters.ts | Adds adapter-provided migration metadata (secret-free manifestConfig). |
| packages/core/src/database/migrations/runner.ts | Adds MIGRATION_NAMES + exact migration status query helper. |
| packages/core/src/database/migrations/policy.ts | Adds migration config normalization + runtime enforcement for auto/check/manual. |
| packages/core/src/database/index.ts | Exposes exact migration status and MIGRATION_NAMES from database exports. |
| packages/core/src/astro/routes/api/setup/index.ts | Removes direct migration running from setup route. |
| packages/core/src/astro/routes/api/setup/dev-bypass.ts | Removes direct migration running from dev-bypass route (runtime now governs). |
| packages/core/src/astro/routes/api/auth/dev-bypass.ts | Removes direct migration running from auth dev-bypass route. |
| packages/core/src/astro/middleware.ts | Resolves effective migration mode and returns 503s for pending/manual-missing-schema cases. |
| packages/core/src/astro/integration/runtime.ts | Adds migrations config to EmDash integration config type. |
| packages/core/src/astro/integration/index.ts | Normalizes migration config; writes migration manifest during build/sync; attaches migration metadata. |
| packages/core/package.json | Exposes new public subpath exports for migration executors and ./migrations. |
| .changeset/runtime-migration-policy.md | Changeset for runtime migration policy modes. |
| .changeset/direct-migration-adapters.md | Changeset for deployment migration executors on built-in adapters. |
| .changeset/deployment-migration-primitives.md | Changeset for public migration identity + exact status APIs. |
| .changeset/deployment-migration-manifest.md | Changeset for manifest emission during build/sync. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
This PR introduces a well-scoped set of deployment-managed migration primitives: a runtime migration policy (auto/check/manual), build-time manifest generation and writing, identity/fingerprinting, and direct SQLite/libSQL/PostgreSQL executors. The approach fits EmDash’s architecture—writer runs at Astro build/sync time, executors are loaded from the same published subpaths used by the runtime, and the runtime enforces the policy without adding queries to the logged-out hot path beyond the existing migration fast-path count.
I read the new migration source, the adapter migration entrypoints, the integration wiring, the runtime policy integration, the Virtual-module type updates, the package exports, tsdown entries, and the test coverage. The code is generally clean, uses Kysely for SQL safety, keeps credentials out of the build artifact, and adds real behavioral tests for check/apply, secret rejection, target fingerprinting, config loading, and manual-mode middleware behavior.
One real logic bug stands out: the direct executor discards a successful migration report if closing the dialect fails. I also have one minor testing nit about a hard-coded SHA fingerprint literal.
Overlapping PRsThis PR modifies files that are also changed by other open PRs:
This may cause merge conflicts or duplicated work. A maintainer will coordinate. |
96ba1b5 to
4ccb0d3
Compare
There was a problem hiding this comment.
This PR adds a well-scoped set of deployment-managed migration primitives: a runtime migration policy (auto/check/manual), build-time manifest generation and writing, migration identity/fingerprinting, and direct SQLite/libSQL/PostgreSQL executors loaded from the same published subpaths the runtime uses. The approach fits EmDash’s architecture—writer runs at Astro build/sync time, the runtime enforces the policy without new logged-out hot-path queries beyond the existing migration fast-path count, and credentials are kept out of the build artifact. It links to the approved Discussion (#2418) and includes changesets, tests, and the necessary package/tsdown entries.
I read the new migration source, the adapter entrypoints, the integration and middleware wiring, the runtime policy integration, the Virtual-module type updates, the package exports, tsdown entries, and the test coverage. This is a static review; I did not run the test suite or build.
Prior-review status:
- ✅ Fixed: the direct executor no longer discards a successful migration report when closing the dialect fails; it now returns the report while logging the close error.
- 🔁 Still open: the hard-coded SHA-256 fingerprint literal in
identity.test.tsremains.
New finding: the middleware’s 503 response tells operators to run emdash migrate, but the EmDash CLI does not expose a migrate subcommand yet. That guidance is currently unfulfilled and will confuse operators when check or manual mode blocks traffic.
Verdict: comment — one real needs_fixing issue (the CLI message), one testing suggestion, and otherwise a clean, deliberate implementation.
0489cd4 to
e450006
Compare
There was a problem hiding this comment.
This is a re-review of the deployment-managed migration primitives PR (#2437). The implementation remains the right shape for #2276/#2418: the build-time manifest keeps secrets out of the artifact, auto/check/manual modes are wired into the existing runtime and middleware path without adding logged-out hot-path queries, and the direct SQLite/libSQL/PostgreSQL executors reuse the same published adapter subpaths as production.
I re-read the migration identity, manifest builder/writer, direct executor, policy, integration wiring, runtime policy integration, middleware 503 paths, and the new/updated tests. Conventions still look good: SQL is parameterized/identifier-safe, the manifest secret-free validator rejects credentials and connection URLs, the setup probe is now gated to auto mode so the query-count snapshot stays unchanged, changesets are present and user-facing, and no admin UI strings were added.
The only unresolved issue from the previous review is still present: the fingerprint test hard-codes a SHA-256 literal. That assertion is a config-pin test that restates the implementation and cannot fail on any real regression other than an intentional algorithm change. The preceding assertions already cover determinism (expect(second).toBe(first)) and hex formatting (expect(first).toMatch(/^[0-9a-f]{64}$/)), so the literal should be removed.
No new security, data-integrity, or logged-out query regressions were introduced.
What does this PR do?
Adds the core primitives for deployment-managed migrations: migration policies, manifests, configuration loading, direct SQLite/libSQL/PostgreSQL executors, runtime enforcement, and package exports. It also adds upgrade, policy, manifest, adapter, and executor coverage.
Part of #2276
Type of change
Checklist
pnpm typecheckpassespnpm lintpassespnpm testpasses (or targeted tests for my change)pnpm formathas been runmessages.pochanges except in translation PRs — a workflow extracts catalogs on merge tomain.Approved design discussion: #2418. Package typechecking, quick lint, and targeted formatting pass. The core suite passes 5,418 tests; its remaining failure is the pre-existing macOS
/varversus/private/varpath-normalization assertion.AI-generated code disclosure
Screenshots / test output
Non-visual change. Core package build and typecheck pass, query-count snapshots are unchanged, and the relevant migration tests pass.