feat(data-warehouse): scaffold google merchant center source - #76410
feat(data-warehouse): scaffold google merchant center source#76410Gilbert09 wants to merge 5 commits into
Conversation
Scaffolds Google Merchant Center as a new data warehouse source stub in response to a user request from the source survey. This is scaffolding only: the source is registered with `unreleasedSource=True` (hidden from users) and has no sync logic yet. Wires up the `ExternalDataSourceType` enum, frontend `externalDataSources` entry, generated config, registration, icon, and a linear Django migration. Generated-By: PostHog Code Task-Id: b9f98a01-e862-45f0-b24e-ac7aff43c837
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
|
Hey @Gilbert09! 👋 It looks like your git author email on this PR isn't your
You can fix it for this repo with: git config user.email "you@posthog.com"Or set it globally with |
🦔 Hogbox preview · 🔧 building…Spinning up PostHog for this PR on a hogland hogbox — this comment updates in place when it's ready (usually a few minutes). commit |
🤖 CI report
|
| Root | Eager (shipped) | Δ vs base | Budget |
|---|---|---|---|
entry (logged-out pages, app bootstrap)src/index.tsx |
1.25 MiB · 22 files | no change | ███░░░░░░░ 27.7% of 4.51 MiB |
authenticated shell (every logged-in page)src/scenes/AuthenticatedShell.tsx |
8.14 MiB · 3,035 files | 🔺 +23 B (+0.0%) | ████████░░ 83.8% of 9.71 MiB |
🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
Largest files eagerly shipped from src/index.tsx
| Size | File |
|---|---|
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 24.6 KiB | ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js |
| 6.3 KiB | ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js |
| 4.5 KiB | ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js |
| 3.9 KiB | ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js |
| 1.4 KiB | ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js |
| 1.3 KiB | src/RootErrorBoundary.tsx |
| 912 B | ../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js |
| 789 B | src/scenes/ChunkLoadErrorBoundary.tsx |
| 762 B | src/index.tsx |
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
| Size | File |
|---|---|
| 285.5 KiB | ../node_modules/.pnpm/posthog-js@1.410.1/node_modules/posthog-js/dist/rrweb.js |
| 267.7 KiB | ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js |
| 235.5 KiB | src/taxonomy/core-filter-definitions-by-group.json |
| 231.5 KiB | ../node_modules/.pnpm/posthog-js@1.410.1/node_modules/posthog-js/dist/module.js |
| 154.3 KiB | ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js |
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 105.3 KiB | src/lib/api.ts |
| 94.7 KiB | ../packages/quill/packages/quill/dist/index.js |
| 93.3 KiB | ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js |
| 90.6 KiB | ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js |
Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479
✅ Toolbar bundle — eager 2.19 MiB within budget
What the toolbar ships to customer pages, measured from the esbuild output (minified, post-tree-shake). The eager set is the entry plus everything statically imported from it — fetched before any feature runs; deferred chunks load lazily. The eager guardrail is 5.72 MiB. Each output file must also stay below 10 MB, where CloudFront stops compressing it. The module boundary is enforced separately by check-toolbar-graph.
| Metric | Size | Δ vs base | Budget |
|---|---|---|---|
| Eager (shipped) entry + static imports |
2.19 MiB · 17 files | no change | ████░░░░░░ 38.3% of 5.72 MiB |
| Deferred (lazy) | 2.08 MiB · 33 files | no change | n/a — loads on demand |
Loader dist/toolbar.js |
1.1 KiB | no change | █░░░░░░░░░ 5.8% of 19.5 KiB |
Largest eagerly-shipped chunks
| Size | File |
|---|---|
| 718.3 KiB | dist/toolbar/toolbar-app-FT4UJ3IP.css |
| 551.4 KiB | dist/toolbar/chunk-chunk-TKZ3SULZ.js |
| 484.6 KiB | dist/toolbar/chunk-chunk-T44C2V5C.js |
| 133.6 KiB | dist/toolbar/chunk-chunk-PETGQ2AR.js |
| 131.8 KiB | dist/toolbar/chunk-chunk-T5KY5WYR.js |
| 71.0 KiB | dist/toolbar/toolbar-app-HOL6RSL7.js |
| 69.0 KiB | dist/toolbar/chunk-chunk-27JL52RE.js |
| 35.6 KiB | dist/toolbar/chunk-chunk-T3UPWJAA.js |
| 20.9 KiB | dist/toolbar/chunk-chunk-YJS276P3.js |
| 12.2 KiB | dist/toolbar/chunk-chunk-PIK3PADE.js |
Posted automatically by check-toolbar-size · sizes are toolbar output bytes (shipped, post-tree-shake) from the esbuild metafile
✅ Dist folder size — 🔺 +17.2 KiB (+0.0%)
Total size of the built frontend/dist folder (all assets), compared against the base branch.
Total: 1383.91 MiB · 🔺 +17.2 KiB (+0.0%)
⚠️ Backend snapshots — 2 updated (2 modified, 0 added, 0 deleted)
Query snapshots: Backend query snapshots updated
Changes: 2 snapshots (2 modified, 0 added, 0 deleted)
What this means:
- Query snapshots have been automatically updated to match current output
- These changes reflect modifications to database queries or schema
Next steps:
- Review the query changes to ensure they're intentional
- If unexpected, investigate what caused the query to change
⚠️ Django migration SQL — 1 new migration to review
We've detected new migrations on this PR. Review the SQL output for each migration:
products/warehouse_sources/backend/migrations/0116_google_merchant_center_source.py
BEGIN;
--
-- Alter field source_type on externaldatasource
--
-- (no-op)
--
-- Alter field source_type on pendingsourcecredential
--
-- (no-op)
COMMIT;Last updated: 2026-08-03 11:48 UTC (b9ba09d)
✅ Django migration risk — migration analysis complete
We've analyzed your migrations for potential risks.
Summary: 0 Safe | 1 Needs Review | 0 Blocked
⚠️ Needs Review
May have performance impact
warehouse_sources.0116_google_merchant_center_source
└─ #1 ⚠️ AlterField
Field alteration may cause table locks or data loss (check if changing type or constraints)
model: externaldatasource, field: source_type, field_type: CharField
└─ #2 ⚠️ AlterField
Field alteration may cause table locks or data loss (check if changing type or constraints)
model: pendingsourcecredential, field: source_type, field_type: CharField
Last updated: 2026-08-03 11:48 UTC (b9ba09d)
There was a problem hiding this comment.
Pull request overview
This PR scaffolds Google Merchant Center as a new (hidden) data warehouse source stub within products/warehouse_sources, wiring it into the existing source registry + generated schema artifacts while ensuring it remains unavailable to users via unreleasedSource=True.
Changes:
- Adds a new
ExternalDataSourceTypeentry for Google Merchant Center and propagates it through generated schema enums/artifacts. - Introduces a new
GoogleMerchantCenterSourcestub (config-only) and registers it in the source loader + source catalog list. - Adds a Django migration updating
source_typechoices for the warehouse-sources models.
Reviewed changes
Copilot reviewed 9 out of 11 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| products/warehouse_sources/backend/types.py | Adds ExternalDataSourceType.GOOGLEMERCHANTCENTER. |
| products/warehouse_sources/backend/temporal/data_imports/sources/SOURCES.md | Adds google_merchant_center to the source scaffold list. |
| products/warehouse_sources/backend/temporal/data_imports/sources/google_merchant_center/source.py | New stub source definition with unreleasedSource=True and E-commerce category. |
| products/warehouse_sources/backend/temporal/data_imports/sources/generated_configs/googlemerchantcenter.py | Generated empty config class for the new source. |
| products/warehouse_sources/backend/temporal/data_imports/sources/_load_all.py | Ensures the new source module is imported/registered. |
| products/warehouse_sources/backend/migrations/max_migration.txt | Advances recorded max migration to 0116_google_merchant_center_source. |
| products/warehouse_sources/backend/migrations/0116_google_merchant_center_source.py | Choices-only migration updating source_type allowed values. |
| posthog/schema_enums.py | Adds schema enum value GOOGLE_MERCHANT_CENTER. |
| frontend/src/queries/schema/schema-general.ts | Adds GoogleMerchantCenter to the externalDataSources list. |
| frontend/src/queries/schema.json | Adds GoogleMerchantCenter to the generated schema JSON. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Add GoogleMerchantCenter to the kea-typegen source-type unions in sourceCatalogLogic and sourceWizardLogic, keeping the generated blocks in sync with the new externalDataSources entry. Fixes the frontend typechecking / schema-up-to-date check. Generated-By: PostHog Code Task-Id: b9f98a01-e862-45f0-b24e-ac7aff43c837
|
👋 Visual changes detected for this PR. Review and approve in PostHog Visual Review If these changes are unexpected, they may be caused by a flaky test or a broken snapshot on master. Don't approve — rerun the job or wait for a fix. |
fuziontech
left a comment
There was a problem hiding this comment.
Automated review agent on behalf of @fuziontech.
The scaffold follows the existing unreleased-source pattern, keeps the connector hidden until implementation, and updates the registry, enums, migration chain, generated configs/types, icon, and kea typegen consistently. Focused source registry/config/category/version tests passed (3,861 tests), along with Ruff and Python compilation checks. No blocking issues found.
…er-source Generated-By: PostHog Code Task-Id: 08bac2d1-a56a-4647-84ad-8dc72c8dd711
Problem
A user asked for Google Merchant Center as a data warehouse source in the source-request survey. We don't have it yet.
Changes
Scaffolds Google Merchant Center as a new source stub, following the
/implementing-warehouse-sourcesskill. This is scaffolding only. The source ships withunreleasedSource=True, so it stays hidden from users and does not yet sync anything.ExternalDataSourceType.GOOGLEMERCHANTCENTERenum value and matchingexternalDataSourcesentry inschema-general.ts(regeneratedposthog/schema_enums.py+schema.jsonviapnpm run schema:build).GoogleMerchantCenterSource(categoryE_COMMERCE) undersources/google_merchant_center/, registered in_load_all.py.pnpm run generate:source-configs.frontend/public/services/google_merchant_center.png(via Logo.dev).0116_google_merchant_center_source(choices-onlyAlterField, no-op SQL) withmax_migration.txtupdated.SOURCES.mdscaffolded-list entry.The other requested survey entry ("yes") was junk, not an identifiable source, so it was skipped.
How did you test this code?
Agent-run automated checks only (no manual/live-API testing):
pnpm run generate:source-configsandpnpm run schema:buildregenerate cleanly.get_source_configresolves (verified viamanage.py shell).makemigrations --check --dry-runreports no changes, single leaf0116,sqlmigrateis a no-op, applied OK locally.pytestover the source suites passes (test_source_categories,test_source_versions,test_source_config_generator,test_all_sources_import,test_source_loading).ruff check/ruff formatclean on touched files.Note
hogli ci:preflight's migration check failed locally with a ClickHouseServerErrorfrom an unhealthy local dev ClickHouse. This change contains no ClickHouse migrations, and the Postgres migration was verified linear through the checks above. Real CI runs against a healthy ClickHouse.Automatic notifications
Docs update
No user-facing docs yet: the source is hidden (
unreleasedSource=True) and has no sync logic. Docs come with the full implementation.🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Authored by Claude (Claude Code). The task was to triage survey-requested warehouse sources and scaffold the legitimate ones per the
/implementing-warehouse-sourcesskill (which I invoked, along with/django-migrations).Decisions:
Created with PostHog Code