Skip to content

Speed up automatic Media Usage indexing - #2539

Draft
khoinguyenpham04 wants to merge 4 commits into
feature/media-usage-activation-uifrom
feature/media-usage-work-conserving-backfill
Draft

Speed up automatic Media Usage indexing#2539
khoinguyenpham04 wants to merge 4 commits into
feature/media-usage-activation-uifrom
feature/media-usage-work-conserving-backfill

Conversation

@khoinguyenpham04

@khoinguyenpham04 khoinguyenpham04 commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

What does this PR do?

Speeds up automatic Media Usage indexing after activation without changing projection, coverage, deletion, or API semantics.

The new work-conserving engine skips idle maintenance classes and processes one existing bounded durable unit per turn. Node continues on event-loop timer turns. Cloudflare uses identity-free Queue wake messages with one consumer at a time, while D1 work rows, leases, retries, and Cron recovery remain authoritative.

This draft is stacked directly on PR #2531. It also includes a small out-of-scope admin fix discovered during deployed testing: incomplete locale catalogs now fall back to compiled English, preventing Media Usage progress from displaying raw plural syntax.

Deployed testing on an isolated D1 database completed 100 historical entries without loss or duplication, including pause and resume. The safe one-unit Queue path indexed first-to-last in about eight minutes; D1 SQL execution itself remained under one second, so Queue delivery latency is the dominant cost.

Draft follow-up before marking ready: the repository lock currently resolves Wrangler 4.100, while first-deploy Queue auto-provisioning was verified to require Wrangler 4.123+.

Related: #2531

Type of change

  • Bug fix
  • Feature (requires maintainer-approved Discussion)
  • Refactor (no behavior change)
  • Translation
  • Documentation
  • Performance improvement
  • Tests
  • Chore (dependencies, CI, tooling)

Checklist

  • I have read CONTRIBUTING.md
  • pnpm typecheck passes
  • pnpm lint passes
  • pnpm test passes (or targeted tests for my change)
  • pnpm format has been run
  • I have added/updated tests for my changes (if applicable)
  • User-visible strings in the admin UI are wrapped for translation (if applicable). Do not include messages.po changes except in translation PRs — a workflow extracts catalogs on merge to main.
  • I have added a changeset (if this PR changes a published package)
  • New features link to an approved Discussion: N/A — scoped follow-up to the approved Media Usage sequence

The locale checklist remains unchecked because this draft deliberately includes one source-catalog entry to reproduce and fix the deployed raw-ICU regression before the normal post-merge extraction workflow runs.

AI-generated code disclosure

  • This PR includes AI-generated code — model/tool: GPT-5.6, Codex

Screenshots / test output

  • Core tests: 5,687 passed; one unrelated network-downloaded WordPress fixture was unavailable locally.
  • Cloudflare package: 338 tests passed.
  • Core workerd/D1: 9 tests passed.
  • Admin browser tests: 1,497 passed.
  • Full package typecheck, type-aware lint, formatting, core/admin/Cloudflare builds, template builds, docs build, Changesets, and Wrangler dry-run passed.
  • Final adversarial whole-diff review: no bugs found.

Try this PR

Open a fresh playground →

A full working EmDash site, deployed from this branch. Each visit gets its own session-scoped sandbox: no login needed and no shared state. Try the admin, edit content, hit the public site.

Tracks feature/media-usage-work-conserving-backfill. Updated automatically when the playground redeploys.

@changeset-bot

changeset-bot Bot commented Aug 18, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 1fcbc86

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 17 packages
Name Type
emdash Patch
@emdash-cms/cloudflare Patch
@emdash-cms/admin Patch
@emdash-cms/sandbox-workerd Patch
@emdash-cms/plugin-mcp-smoke Patch
@emdash-cms/fixture-perf-site Patch
@emdash-cms/perf-demo-site Patch
@emdash-cms/cache-demo-site Patch
@emdash-cms/do-demo-site Patch
@emdash-cms/do-solo-demo-site Patch
@emdash-cms/auth Patch
@emdash-cms/blocks Patch
@emdash-cms/gutenberg-to-portable-text Patch
@emdash-cms/x402 Patch
create-emdash Patch
@emdash-cms/auth-atproto Patch
@emdash-cms/plugin-embeds Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 18, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
emdash-demo-do 1fcbc86 Aug 18 2026, 10:43 AM

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 18, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
emdash-playground 1fcbc86 Aug 18 2026, 10:43 AM

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 18, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
emdash-demo-cache 1fcbc86 Aug 18 2026, 10:43 AM

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
docs 1fcbc86 Aug 18 2026, 10:01 AM

@github-actions

Copy link
Copy Markdown
Contributor

Scope check

This PR changes 1,326 lines across 39 files. Large PRs are harder to review and more likely to be closed without review.
This PR spans 5 different areas (area/core, area/admin, area/docs, area/templates, area/cloudflare). Consider breaking it into smaller, focused PRs.

If this scope is intentional, no action needed. A maintainer will review it. If not, please consider splitting this into smaller PRs.

See CONTRIBUTING.md for contribution guidelines.

@github-actions

Copy link
Copy Markdown
Contributor

Lunaria Status Overview

🌕 This pull request will trigger status changes.

Learn more

By default, every PR changing files present in the Lunaria configuration's files property will be considered and trigger status changes accordingly.

You can change this by adding one of the keywords present in the ignoreKeywords property in your Lunaria configuration file in the PR's title (ignoring all files) or by including a tracker directive in the merged commit's description.

Tracked Files

File Note
packages/admin/src/locales/en/messages.po Source changed, localizations will be marked as outdated.
Warnings reference
Icon Description
🔄️ The source for this localization has been updated since the creation of this pull request, make sure all changes in the source have been applied.

@github-actions

Copy link
Copy Markdown
Contributor

Overlapping PRs

This PR modifies files that are also changed by other open PRs:

This may cause merge conflicts or duplicated work. A maintainer will coordinate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant