Skip to content

fix(core): add controlled media usage activation - #2445

Open
khoinguyenpham04 wants to merge 24 commits into
mainfrom
feature/media-usage-production-activation
Open

fix(core): add controlled media usage activation#2445
khoinguyenpham04 wants to merge 24 commits into
mainfrom
feature/media-usage-production-activation

Conversation

@khoinguyenpham04

@khoinguyenpham04 khoinguyenpham04 commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

What does this PR do?

Adds a deliberate, administrator-controlled way to turn on automatic Media Usage indexing after a production deployment is ready.

The indexing foundation stays dormant until an administrator explicitly starts activation. This PR provides that final safety gate:

  • administrators can read a redacted activation status without exposing internal lease tokens or database errors;
  • each confirmed request activates at most one collection, so work stays bounded and can safely resume after interruption;
  • activation requires schema:manage; bearer-token calls need the admin scope, session-based POSTs need CSRF protection, and every POST requires explicit confirmation that writers are stopped and background maintenance is ready;
  • existing leases, retries, trigger verification, and fail-closed writer protection remain responsible for correctness;
  • activation never starts automatically, never runs historical reconciliation in the request, and never claims that historical coverage is already complete;
  • the TypeScript client, OpenAPI reference, REST documentation, and Cloudflare and Node deployment guides explain the same operator workflow;
  • V1 behavior and logged-out query counts remain unchanged.

An active result means that new content changes are being captured. Historical content continues to catch up through the bounded background reconciliation added by #2443.

This is stacked directly on #2443 and should be reviewed relative to feature/media-usage-automatic-reconciliation.

Type of change

  • Bug fix
  • Feature (requires maintainer-approved Discussion)
  • Refactor (no behavior change)
  • Translation
  • Documentation
  • Performance improvement
  • Tests
  • Chore (dependencies, CI, tooling)

Checklist

  • I have read CONTRIBUTING.md
  • pnpm typecheck passes
  • pnpm lint passes
  • pnpm test passes (or targeted tests for my change)
  • pnpm format has been run
  • I have added/updated tests for my changes (if applicable)
  • User-visible strings in the admin UI are wrapped for translation (not applicable; this PR adds no admin UI or user-facing UI strings)
  • I have added a changeset (if this PR changes a published package)
  • New features link to an approved Discussion: not applicable; this completes controlled activation for the existing Media Usage indexing system

AI-generated code disclosure

  • This PR includes AI-generated code — model/tool: OpenAI Codex (GPT-5) with GPT-5.6 Terra adversarial review

Screenshots / test output

No visual changes.

Verified locally:

  • full core suite: 5,620 passed, 5 skipped;
  • real D1/workerd activation suite: 8 passed;
  • fresh focused API, OpenAPI, and client suite: 75 passed;
  • typecheck, type-aware lint, quick lint, formatting, core and documentation builds, OpenAPI validation, and Changesets validation;
  • SQLite and D1 logged-out query counts and SQL snapshots are unchanged;
  • the worst measured activation batch used 21 D1 queries, below the 40-query safety gate;
  • a final GPT-5.6 Terra xhigh adversarial review found no bugs.

PostgreSQL and Durable Object SQL runtime tests were not available locally. Their shared code paths are typechecked, while the available SQLite and real D1/workerd tests cover the public activation contract and bounded request behavior.


Try this PR

Open a fresh playground →

A full working EmDash site, deployed from this branch. Each visit gets its own session-scoped sandbox: no login needed and no shared state. Try the admin, edit content, hit the public site.

Tracks feature/media-usage-production-activation. Updated automatically when the playground redeploys.

…utomatic-reconciliation

# Conflicts:
#	.changeset/safe-collection-deletion.md
#	packages/core/src/database/migrations/runner.ts
#	packages/core/src/database/repositories/media-usage.ts
#	packages/core/src/database/types.ts
#	packages/core/src/emdash-runtime.ts
#	packages/core/src/media/usage/collection-deletion-processor.ts
#	packages/core/src/media/usage/collection-deletion.ts
#	packages/core/tests/integration/database/media-usage-incremental-work-migration.test.ts
#	packages/core/tests/integration/database/migrations.test.ts
#	packages/core/tests/integration/runtime/media-usage-scheduled-driver.test.ts
#	packages/core/tests/workerd/media-usage-collection-deletion-d1.test.ts
@changeset-bot

changeset-bot Bot commented Aug 12, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: c862530

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 17 packages
Name Type
emdash Patch
@emdash-cms/cloudflare Patch
@emdash-cms/sandbox-workerd Patch
@emdash-cms/plugin-mcp-smoke Patch
@emdash-cms/fixture-perf-site Patch
@emdash-cms/perf-demo-site Patch
@emdash-cms/cache-demo-site Patch
@emdash-cms/do-demo-site Patch
@emdash-cms/do-solo-demo-site Patch
@emdash-cms/admin Patch
@emdash-cms/auth Patch
@emdash-cms/blocks Patch
@emdash-cms/gutenberg-to-portable-text Patch
@emdash-cms/x402 Patch
create-emdash Patch
@emdash-cms/auth-atproto Patch
@emdash-cms/plugin-embeds Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

Copy link
Copy Markdown
Contributor

Scope check

This PR changes 1,113 lines across 16 files. Large PRs are harder to review and more likely to be closed without review.

If this scope is intentional, no action needed. A maintainer will review it. If not, please consider splitting this into smaller PRs.

See CONTRIBUTING.md for contribution guidelines.

@khoinguyenpham04 khoinguyenpham04 added the bot:review Trigger an emdashbot code review on this PR label Aug 12, 2026

@emdashbot emdashbot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a sound follow-up to #2443: it adds a deliberately gated admin API to turn on Media Usage indexing, which is exactly the safety gate the system needs before it starts capturing real writes. The implementation follows EmDash's conventions well — thin route wrappers, handler-layer ApiResult, schema:manage plus admin token scope, CSRF via the existing middleware, redacted status responses, OpenAPI/client updates, and D1 query-budget tests. Logged-out routes are untouched.

I checked the diff, the full new route/handler/core changes, and relevant siblings. The code looks correct: the status endpoint omits lease tokens and raw error values, the POST requires both literal confirmations via Zod, the lease/conflict/version-mismatch errors map cleanly to 409s, and the test suite covers auth, redaction, idempotency, lease races, and failure paths.

One small AGENTS-style cleanup: the registration test in packages/core/tests/unit/api/media-usage-activation-route.test.ts is a config-pin test — it asserts the literal route pattern back at the source. It cannot fail on a real regression and will break whenever the route is intentionally moved or renamed. It should be removed; the behavior tests that actually call the route already prove it is wired up correctly.

Comment thread packages/core/tests/unit/api/media-usage-activation-route.test.ts
@emdashbot emdashbot Bot removed the bot:review Trigger an emdashbot code review on this PR label Aug 12, 2026
@github-actions github-actions Bot added review/awaiting-author Reviewed; waiting on the author to respond and removed review/needs-review No maintainer or bot review yet labels Aug 12, 2026

@emdashbot emdashbot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the full diff statically and traced the new activation endpoints through the handler, route, business logic, OpenAPI schema, client, tests, docs, and the auth/write-fence middleware.

Approach judgment: This is a reasonable completion of the Media Usage indexing foundation. It adds a deliberate, admin-controlled activation gate rather than auto-activating, which matches the PR description and the existing architecture. The API envelope, authorization (schema:manage + admin scope for bearer tokens), and CSRF handling (middleware-enforced for session POSTs) all line up with EmDash conventions. The change is additive and does not touch logged-out paths, so query-count snapshots should be unaffected.

What I checked:

  • Error-code registration and HTTP-status mapping for the new activation codes.
  • Handler-to-route wiring, including requirePerm, requireScope, parseBody, and unwrapResult.
  • Schema validation (writersDrained: true and maintenanceReady: true literals, .strict()).
  • The activation business logic: lease claiming, bounded batch processing (≤1 collection/call), idempotent active state, conflict/error paths, and redacted status reads.
  • Trigger installation is idempotent via capture-triggers.ts, so retries after partial installs should not get stuck.
  • OpenAPI additions and client method wiring.
  • New unit and workerd test coverage.
  • Docs links/anchors and changeset.
  • No SQL interpolation issues, no new content-table queries missing locale filters, no new logged-out queries.

Headline conclusion: The code looks correct and well-scoped. I did not find any blocking issues, logic bugs, AGENTS.md convention violations, or missing authorization/CSRF checks. Runtime test/lint claims in the PR description are noted but not independently verified.

One non-blocking design note: maintenanceReady is enforced only at the route schema boundary and is not forwarded into activateMediaUsageCapture. That is acceptable because the route is the intended entry point and the confirmation is operator-level, but you may want to thread it to the business-function boundary for symmetry if internal callers ever invoke activation directly.

@github-actions github-actions Bot added review/approved Approved; no new commits since and removed review/awaiting-author Reviewed; waiting on the author to respond labels Aug 12, 2026
Comment thread docs/src/content/docs/deployment/cloudflare.mdx Outdated

@ascorbic ascorbic left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code seems fine. The docs though have really weird LLM wording that's very unclear to anyone who doesn't know the internals of EmDash. It needs some serious rewriting with a user-facing voice.

Comment thread docs/src/content/docs/deployment/nodejs.mdx Outdated
Comment thread docs/src/content/docs/reference/rest-api.mdx Outdated
Comment thread docs/src/content/docs/reference/rest-api.mdx Outdated
Comment thread docs/src/content/docs/reference/rest-api.mdx Outdated
Base automatically changed from feature/media-usage-automatic-reconciliation to main August 13, 2026 12:29
@github-actions

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot added review/needs-rereview Author pushed changes since the last review area/cloudflare area/templates and removed review/approved Approved; no new commits since labels Aug 14, 2026
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 14, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
docs c862530 Aug 14 2026, 03:20 PM

@pkg-pr-new

pkg-pr-new Bot commented Aug 14, 2026

Copy link
Copy Markdown

Open in StackBlitz

@emdash-cms/admin

npm i https://pkg.pr.new/@emdash-cms/admin@2445

@emdash-cms/auth

npm i https://pkg.pr.new/@emdash-cms/auth@2445

@emdash-cms/auth-atproto

npm i https://pkg.pr.new/@emdash-cms/auth-atproto@2445

@emdash-cms/blocks

npm i https://pkg.pr.new/@emdash-cms/blocks@2445

@emdash-cms/cloudflare

npm i https://pkg.pr.new/@emdash-cms/cloudflare@2445

@emdash-cms/contentful-to-portable-text

npm i https://pkg.pr.new/@emdash-cms/contentful-to-portable-text@2445

emdash

npm i https://pkg.pr.new/emdash@2445

create-emdash

npm i https://pkg.pr.new/create-emdash@2445

@emdash-cms/gutenberg-to-portable-text

npm i https://pkg.pr.new/@emdash-cms/gutenberg-to-portable-text@2445

@emdash-cms/plugin-cli

npm i https://pkg.pr.new/@emdash-cms/plugin-cli@2445

@emdash-cms/plugin-types

npm i https://pkg.pr.new/@emdash-cms/plugin-types@2445

@emdash-cms/registry-client

npm i https://pkg.pr.new/@emdash-cms/registry-client@2445

@emdash-cms/registry-lexicons

npm i https://pkg.pr.new/@emdash-cms/registry-lexicons@2445

@emdash-cms/registry-verification

npm i https://pkg.pr.new/@emdash-cms/registry-verification@2445

@emdash-cms/sandbox-workerd

npm i https://pkg.pr.new/@emdash-cms/sandbox-workerd@2445

@emdash-cms/x402

npm i https://pkg.pr.new/@emdash-cms/x402@2445

@emdash-cms/plugin-ai-moderation

npm i https://pkg.pr.new/@emdash-cms/plugin-ai-moderation@2445

@emdash-cms/plugin-atproto

npm i https://pkg.pr.new/@emdash-cms/plugin-atproto@2445

@emdash-cms/plugin-audit-log

npm i https://pkg.pr.new/@emdash-cms/plugin-audit-log@2445

@emdash-cms/plugin-color

npm i https://pkg.pr.new/@emdash-cms/plugin-color@2445

@emdash-cms/plugin-embeds

npm i https://pkg.pr.new/@emdash-cms/plugin-embeds@2445

@emdash-cms/plugin-field-kit

npm i https://pkg.pr.new/@emdash-cms/plugin-field-kit@2445

@emdash-cms/plugin-forms

npm i https://pkg.pr.new/@emdash-cms/plugin-forms@2445

@emdash-cms/plugin-webhook-notifier

npm i https://pkg.pr.new/@emdash-cms/plugin-webhook-notifier@2445

commit: c862530

@github-actions github-actions Bot added review/approved Approved; no new commits since and removed review/needs-rereview Author pushed changes since the last review labels Aug 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants