feat(import): complete WordPress migrations via the exporter plugin (taxonomies, ACF, SEO, menus, comments, site identity, link rewriting)#1830
Conversation
…, menus, media pagination, rest_route fallback, migration key) - execute.ts now pre-creates taxonomy terms via the shared WXR taxonomy machinery and attaches category/tag/custom-taxonomy assignments per post - Yoast/Rank Math per-post titles and descriptions land in seo_title / seo_description fields (auto-created like other import fields) - ACF values and custom meta are written to schema fields with matching slugs (created by the prepare step); no fields are invented - Navigation menus are imported after content via the plugin's /menus endpoint, resolving item references through a WP-ID -> EmDash-ID map - analyze() paginates the media list instead of stopping at 500 items - All plugin API calls fall back to ?rest_route= when /wp-json/ 404s (plain permalinks) - Admin import screen accepts an em1. migration key generated by the EmDash Exporter plugin wizard (URL + credentials in one paste)
… WordPress plugin - Import approved/pending comments with authors, original dates, threading, and status into _emdash_comments (idempotent re-runs). - Take over site identity (title, tagline, logo, favicon) from /options, writing the real site:* settings keys and side-loading logo/favicon media. - Surface per-post-type custom fields (ACF and plain meta) from the plugin analysis as suggested collection fields, with sanitized slugs and inferred types; match incoming meta keys with the same sanitization on execute. - Fix field auto-creation to run per field and item instead of once per collection, so fields needed only by a later item (e.g. a lone Yoast SEO override) are still created.
…internal link rewriting - Map plugin custom_fields into requiredFields during analysis (with sanitizeFieldSlug so hyphenated meta keys like event-date become valid field slugs) and sanitize incoming meta keys on execute so values land - Filter well-known plugin bookkeeping meta (rank_math_*, wpil_*, ...) out of field suggestions - Ensure import fields per (collection, field) instead of once per collection, so items later in the stream still get their SEO fields - Coerce meta values to the target field type; drop incoercible values instead of failing the whole item - Read created content IDs from data.item.id (was data.id, which broke the WP-ID map and skipped all comments and taxonomy assignments) - Auto-create taxonomy defs for custom CPT taxonomies during plugin import, scoped to the mapped collections (plugin now exports post_types per taxonomy) - Rewrite internal links in imported content to root-relative URLs in both plugin and WXR sources, so posts stop linking back to the old WordPress domain
🦋 Changeset detectedLatest commit: 687bab1 The changes in this PR will be included in the next version bump. This PR includes changesets to release 16 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Scope checkThis PR changes 2,412 lines across 16 files. Large PRs are harder to review and more likely to be closed without review. If this scope is intentional, no action needed. A maintainer will review it. If not, please consider splitting this into smaller PRs. See CONTRIBUTING.md for contribution guidelines. |
@emdash-cms/admin
@emdash-cms/auth
@emdash-cms/auth-atproto
@emdash-cms/blocks
@emdash-cms/cloudflare
@emdash-cms/contentful-to-portable-text
emdash
create-emdash
@emdash-cms/gutenberg-to-portable-text
@emdash-cms/plugin-cli
@emdash-cms/plugin-types
@emdash-cms/registry-client
@emdash-cms/registry-lexicons
@emdash-cms/sandbox-workerd
@emdash-cms/x402
@emdash-cms/plugin-ai-moderation
@emdash-cms/plugin-atproto
@emdash-cms/plugin-audit-log
@emdash-cms/plugin-color
@emdash-cms/plugin-embeds
@emdash-cms/plugin-field-kit
@emdash-cms/plugin-forms
@emdash-cms/plugin-webhook-notifier
commit: |
![emdashbot[bot]](https://avatars.githubusercontent.com/in/3255304?s=60&v=4){kind=small}
There was a problem hiding this comment.
This is a substantial, well-scoped improvement to WordPress plugin imports that closes real migration gaps (CPT taxonomies, ACF/meta, SEO, menus, comments, site identity, link rewriting, media pagination, plain-permalink fallback, and the migration-key UX). The approach reuses existing EmDash machinery (WXR taxonomy helpers, handleContentCreate, importMediaWithProgress) rather than inventing parallel code, which is good architecture.
The main caveat is process: AGENTS.md requires an approved Discussion for new features, and the PR description acknowledges it doesn't have one. The author is transparent about this and offers to open one, so I won't block on process alone, but the maintainer should confirm whether this needs a Discussion before merge.
Code-wise I found two concrete issues that should be fixed:
-
Migration key decoding ignores base64url padding. The EmDash Exporter key is described as
em1.+ base64url of a JSON blob. Standard base64url is unpadded, butatobrequires padding. Without adding padding, many legitimate keys will decode-fail and the UI will tell the user the key is invalid. -
Raw-HTML link relativization only handles double-quoted
hrefs. Thehref="..."regex inrelativizeContentLinksmisses single-quoted and unquoted attributes, so HTML blocks containing internal links with non-double-quotedhrefs will keep pointing at the old WordPress domain after import, partially defeating the stated goal.
Tests, changeset, Lingui wrapping, and RTL-safe classes all look correct.
| function decodeMigrationKey(key: string): { url: string; user: string; pass: string } | null { | ||
| if (!key.startsWith(MIGRATION_KEY_PREFIX)) return null; | ||
| try { | ||
| const base64 = key.slice(MIGRATION_KEY_PREFIX.length).replace(/-/g, "+").replace(/_/g, "/"); | ||
| const parsed: unknown = JSON.parse(atob(base64)); |
There was a problem hiding this comment.
[needs fixing] decodeMigrationKey converts base64url characters back to base64 but does not restore padding. atob requires the input length to be a multiple of four, so a standard unpadded base64url payload (e.g. a realistic JSON migration key) will throw and the UI will reject a valid key.
| function decodeMigrationKey(key: string): { url: string; user: string; pass: string } | null { | |
| if (!key.startsWith(MIGRATION_KEY_PREFIX)) return null; | |
| try { | |
| const base64 = key.slice(MIGRATION_KEY_PREFIX.length).replace(/-/g, "+").replace(/_/g, "/"); | |
| const parsed: unknown = JSON.parse(atob(base64)); | |
| function decodeMigrationKey(key: string): { url: string; user: string; pass: string } | null { | |
| if (!key.startsWith(MIGRATION_KEY_PREFIX)) return null; | |
| try { | |
| let base64 = key | |
| .slice(MIGRATION_KEY_PREFIX.length) | |
| .replace(/-/g, "+") | |
| .replace(/_/g, "/"); | |
| base64 = base64.padEnd(base64.length + ((4 - (base64.length % 4)) % 4), "="); | |
| const parsed: unknown = JSON.parse(atob(base64)); | |
| // ... |
There was a problem hiding this comment.
Padding restored in 5464331. One factual note for the record: atob implements WHATWG forgiving-base64, which accepts unpadded input — only a remainder-of-1 length fails, and that's impossible for valid base64. That's why every live-tested key (all generated unpadded by the plugin's rtrim(base64_encode(...), '=')) decoded fine. Added the padEnd anyway so strict polyfills or non-browser environments can't trip on it — it costs one line.
| const hrefPattern = new RegExp( | ||
| `href="https?://(?:www\\.)?${sourceHost.replace(REGEX_SPECIALS, "\\$&")}(/[^"]*)?"`, | ||
| "g", | ||
| ); | ||
|
|
||
| for (const block of blocks) { | ||
| switch (block._type) { | ||
| case "block": | ||
| relativizeMarkDefs(block.markDefs, sourceHost); | ||
| break; | ||
| case "image": | ||
| // asset.url stays absolute (media pass), only the click-through link | ||
| if (block.link) block.link = relativizeUrl(block.link, sourceHost) ?? block.link; | ||
| break; | ||
| case "table": | ||
| for (const row of block.rows) { | ||
| for (const cell of row.cells) relativizeMarkDefs(cell.markDefs, sourceHost); | ||
| } | ||
| break; | ||
| case "columns": | ||
| for (const column of block.columns) relativizeContentLinks(column.content, siteUrl); | ||
| break; | ||
| case "cover": | ||
| relativizeContentLinks(block.content, siteUrl); | ||
| break; | ||
| case "button": | ||
| if (block.url) block.url = relativizeUrl(block.url, sourceHost) ?? block.url; | ||
| break; | ||
| case "buttons": | ||
| for (const button of block.buttons) { | ||
| if (button.url) button.url = relativizeUrl(button.url, sourceHost) ?? button.url; | ||
| } | ||
| break; | ||
| case "htmlBlock": | ||
| block.html = block.html.replace(hrefPattern, (_m, path: string | undefined) => { |
There was a problem hiding this comment.
[needs fixing] The htmlBlock handler only rewrites href="..." links. HTML blocks in WordPress commonly use single-quoted or unquoted attributes, so those internal links will remain absolute and keep sending readers to the old domain.
Use a more permissive pattern (or parse and walk the HTML) to catch href='...' and href=... forms as well. For example:
| const hrefPattern = new RegExp( | |
| `href="https?://(?:www\\.)?${sourceHost.replace(REGEX_SPECIALS, "\\$&")}(/[^"]*)?"`, | |
| "g", | |
| ); | |
| for (const block of blocks) { | |
| switch (block._type) { | |
| case "block": | |
| relativizeMarkDefs(block.markDefs, sourceHost); | |
| break; | |
| case "image": | |
| // asset.url stays absolute (media pass), only the click-through link | |
| if (block.link) block.link = relativizeUrl(block.link, sourceHost) ?? block.link; | |
| break; | |
| case "table": | |
| for (const row of block.rows) { | |
| for (const cell of row.cells) relativizeMarkDefs(cell.markDefs, sourceHost); | |
| } | |
| break; | |
| case "columns": | |
| for (const column of block.columns) relativizeContentLinks(column.content, siteUrl); | |
| break; | |
| case "cover": | |
| relativizeContentLinks(block.content, siteUrl); | |
| break; | |
| case "button": | |
| if (block.url) block.url = relativizeUrl(block.url, sourceHost) ?? block.url; | |
| break; | |
| case "buttons": | |
| for (const button of block.buttons) { | |
| if (button.url) button.url = relativizeUrl(button.url, sourceHost) ?? button.url; | |
| } | |
| break; | |
| case "htmlBlock": | |
| block.html = block.html.replace(hrefPattern, (_m, path: string | undefined) => { | |
| const hrefPattern = new RegExp( | |
| `href=["']?https?://(?:www\\.)?${sourceHost.replace(REGEX_SPECIALS, "\\$&")}(/[^"'\\s>]*)?["']?`, | |
| "gi", | |
| ); | |
| // ... | |
| case "htmlBlock": | |
| block.html = block.html.replace(hrefPattern, (match, path: string | undefined) => { | |
| return match.replace(/href=["']?[^"'\\s>]+/i, `href="${path || "/"}"`); | |
| }); | |
| break; |
There was a problem hiding this comment.
Good catch — fixed in 5464331. The pattern now matches double-quoted, single-quoted, and unquoted href values (via a backreference for the closing quote) and is case-insensitive for HREF=. Rewritten links are normalized to href="...". Went with a single regex with a backreference instead of the suggested two-step replace — same coverage, no nested replace. Test extended with all three quote styles plus an external link that must stay untouched.
github-actions
Bot
added
review/awaiting-author
…ef quote styles - Restore base64 padding in decodeMigrationKey before atob. Browsers' forgiving-base64 accepts unpadded input (verified: remainder-1 lengths are impossible for valid base64), but strict polyfills may not. - relativizeContentLinks htmlBlock handler now matches single-quoted, unquoted, and uppercase href attributes, not just href="...".
github-actions
Bot
added
review/needs-rereview
github-actions
Bot
added
review/approved
2 participants
What does this PR do?
Makes migrating from WordPress dramatically easier and more complete. Today, the plugin import path silently drops most of a site: taxonomy assignments, ACF/custom fields, SEO metadata, menus, comments, site identity, and everything past the first 500 media files. A user who migrates discovers the gaps only after switching. This PR closes those gaps end-to-end, together with its companion plugin PR (emdash-cms/wp-emdash#8) — the plugin exports a complete site, and this PR makes EmDash import all of it.
What now migrates (plugin import path):
companytaxonomy on acompanyCPT) get their EmDash taxonomy definitions auto-created, scoped to the collections the post types map onto — previously they were dropped as "missing".rank_math_*/wpil_*filtered out) and populated on import. Hyphenated meta keys (event-date) are sanitized into valid field slugs consistently on both the create and the match side. Meta values are coerced to the target field type (WP meta is stringly-typed and inconsistent across posts); incoercible values are dropped instead of failing the item.seo_title/seo_descriptionfields — ensured per (collection, field) so items later in the stream still get them.?rest_route=fallback (no routes mapping in lokal installations wp-emdash#5, Doesn't seem to detect sites managed with Local wp-emdash#7 territory).Also fixes a bug where created content IDs were read from the wrong response field, which broke the WP-ID map and silently skipped all comments and taxonomy assignments.
Verified end-to-end against two live WordPress sites: a small business site (CPTs + ACF + Yoast) and a ~1.8k-post blog with 3.5k media files, 300+ comments, Rank Math, and custom taxonomies — imported with zero content errors.
Related: emdash-cms/wp-emdash#2 (partially — CPT taxonomies now migrate), emdash-cms/wp-emdash#3 (hyphenated slugs), #1691 (this was developed against exactly such realistic live migrations).
Type of change
Checklist
pnpm typecheckpassespnpm lintpassespnpm testpasses (or targeted tests for my change) — full import suites green (229 tests); known local Node 24.1.0vite-config.test.tsfailures are unrelated (admin build ESM issue), deferring to CIpnpm formathas been runmessages.pochanges except in translation PRs — a workflow extracts catalogs on merge tomain.AI-generated code disclosure
Screenshots / test output
Import summary after the large-blog stress test: 1768 content items, 309 comments, 1689 taxonomy assignments, menus, and site identity imported with 0 errors.