-
Notifications
You must be signed in to change notification settings - Fork 15
Add Site→Project and locale migration script #1139
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
packages/spacecat-shared-data-access/scripts/migrate-sites-to-projects-and-locales.mjs
Fixed
Show fixed
Hide fixed
|
This PR will trigger a patch release when merged. |
packages/spacecat-shared-data-access/scripts/migrate-sites-to-projects-and-locales.mjs
Fixed
Show fixed
Hide fixed
packages/spacecat-shared-data-access/scripts/migrate-sites-to-projects-and-locales.mjs
Outdated
Show resolved
Hide resolved
| const effectiveConcurrency = Number.isFinite(concurrency) && concurrency > 0 | ||
| ? concurrency | ||
| : 10; | ||
| await mapWithConcurrency(sites, effectiveConcurrency, processOne); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would highly recommend to not use any concurrency in this script. Since the script should re-use projects for sites onboarded with multiple locales (e.g. bulk.com/uk and bulk.com/fr should both use project bulk.com), you might run into issues when both sites are created at the same time while the database wasn't updated yet.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done, updated the script to not use concurrency.
For each Site:
If projectId is missing:
Script Usage :
Table selection via env:
export DYNAMO_TABLE_NAME_DATA=spacecat-services-data # or any Spacecat table export AWS_REGION=us-east-1Dry run (no writes, with report):
node packages/spacecat-shared-data-access/scripts/migrate-sites-to-projects-and-locales.mjs \ --orgId=<ORG_ID> \ --report \ --dry-run \ --concurrency=10 \ --no-networkApply (perform writes, with report):
node packages/spacecat-shared-data-access/scripts/migrate-sites-to-projects-and-locales.mjs \ --orgId=<ORG_ID> \ --apply \ --yes \ --report \ --concurrency=10 \ --no-networkSupported filters: --orgId, --projectId, --domain, --siteId. Reports are written to packages/spacecat-shared-data-access/reports/ as CSV and JSON with columns: siteId, organizationId, baseURL, projectId, language, region, action.